Non-response to a life course socioeconomic position indicator in surveillance: comparison of telephone and face-to-face modes

Abstract

Background

Measurement of socioeconomic position (SEP) over the life course in population health surveillance systems is important for examining differences in health and illness between different population groups and for monitoring the impact of policies and interventions aimed at reducing health inequities and intergenerational disadvantage over time. While face-to-face surveys are considered the gold standard of interviewing techniques, computer-assisted telephone interviewing is often preferred for cost and convenience. This study compared recall of parents' highest level of education in telephone and face-to-face surveys.

Methods

Questions about father's and mother's highest education level were included in two representative population health surveys of South Australians aged 18 years and over in Spring 2004. A random sample selected from the electronic white pages (EWP) responded to a computer-assisted telephone interview (n = 2999), and a multistage clustered area sample responded to a face-to-face interview (n = 2893). A subsample of respondents in the face-to-face sample who owned a telephone that was listed in the EWP (n = 2206) was also compared to the telephone interview sample.

Results

The proportion of respondents who provided information about their father's and mother's highest education level was significantly higher in the face-to-face interview (86.3% and 87.8%, respectively) than in the telephone interview (80.4% and 79.9%, respectively). Recall was also significantly higher in the subsample of respondents in the face-to-face interview who had a telephone that was listed in the EWP. Those with missing data for parents' education were more likely to be socioeconomically disadvantaged regardless of the survey mode.

Conclusion

While face-to-face interviewing obtained higher item response rates for questions about parents' education, survey mode did not appear to influence the factors associated with having missing data on father's or mother's highest education level.

Background

Telephone and face-to-face modalities are commonly used in surveillance systems to obtain information about the health and socioeconomic position (SEP) of the population over time. Face-to-face interviews have traditionally been considered the gold standard [1, 2] because of their ability to obtain high unit and item response rates [3, 4] and valid data [5]. Detailed information can also be captured in face-to-face interviews because longer interviews can be conducted [3] and support can be provided by interviewers as respondents observe non-verbal expressions [6]. Telephone survey methods are often preferred over face-to-face methods because of the convenience of computer-assisted telephone interviewing (CATI) and the increasing costs of conducting face-to-face surveys [7]. Telephone surveys are quicker, less costly, less intrusive to respondents, have comparable response rates, and are easier to administer and maintain quality control with monitoring of interviewers than face-to-face surveys [3, 5, 8, 9]. Agreement between telephone and face-to-face surveys in the prevalence of health conditions and risk factors has also been demonstrated [8, 10].

Surveillance systems traditionally measure current SEP among adults, but do not collect information about respondents' early life SEP. SEP refers to both the social and material resources that influence the position people occupy in society [11, 12]. There is increasing evidence about the association of SEP over the life course and health in adulthood [13, 14]. Many indicators can be used to measure early life SEP [15]. Mother's and father's highest level of education are commonly used indicators of early life SEP because they are considered direct measures of the status-based construct of SEP that remains relatively stable during adulthood. Inclusion of such questions in surveillance systems to obtain information about early life SEP will enable monitoring of the impact of policies designed to alter trajectories based on early life SEP. It will also allow a more nuanced assessment of the role of SEP. For example, the ability to monitor trends in health status between groups with different life course experiences – those who are disadvantaged in early life and remain disadvantaged through adulthood, those who are disadvantaged in early life but advantaged during adulthood (upward mobility), those who experience downward mobility from childhood to adulthood, and those who are advantaged during both early life and adulthood. In particular, surveillance systems could then assess the impact of policies designed to counter the impact of disadvantage.

Surveillance systems are different from longitudinal cohort studies. They do not provide aetiological evidence that low SEP during childhood causes poor health outcomes in adulthood. The trend data they can provide are necessary, however, to monitor the effectiveness of policies and programs, such as those related to education, employment, early childhood development and those aimed at changing health-related behaviours among the population, and specifically whether such policies have a differential effect on groups with different socioeconomic life course experiences. Conducting such trend analyses among specific age cohort strata would also provide insights into the effects of historical programs and policies among the various social mobility groups. In addition, health promotion campaigns targeted at the general population may be more successful among advantaged groups who may be more health literate. But are such campaigns, programs, or policies least successful among those who have experienced cumulative disadvantage over their lifetime? Are they more successful among those who were disadvantaged in early life but are more advantaged in adulthood? Answers to these questions can be provided in a timely and relatively inexpensive way, by well-designed surveillance systems.

To obtain information from respondents about their early life SEP, surveillance systems necessarily rely on retrospectively recalled information. It is unknown whether survey mode influences item non-response to questions related to early life SEP. The aim of this paper is to compare the extent of missing data to questions about parents' highest education level obtained retrospectively from similar telephone and face-to-face representative population surveys.

Methods

Telephone survey

In the Health Monitor telephone survey conducted by the South Australian Department of Health in September 2004, households were randomly selected from those in South Australia with a telephone connected and the number listed in the Electronic White Pages (EWP). The person aged 18 years or over who was last to have a birthday was selected for interview. Up to six call backs were made in an attempt to interview them as selected persons were non-replaceable. A letter was sent to each selected household prior to interview, to inform about the purpose of the survey. CATI was used to conduct the interviews. From the eligible sample (n = 4342), 714 refused to participate in the interview, 233 were not contactable after six attempts, 160 were incapacitated, 135 were unavailable for interview, 90 were excluded due to language barriers (interviews were only conducted in English, Greek, Italian and Vietnamese), and eight terminated the interview. An additional three respondents were excluded due to missing data required for weighting purposes. A total of 2999 completed interviews were analysed, a response rate of 69.1%.

Face-to-face survey

The South Australian Health Omnibus Survey is a systematic, self-weighting, multistage, clustered area sample of the metropolitan area and regional centres with a population greater than 1,000. Hotels, motels, hospitals, nursing homes and other major institutions are excluded. To achieve a sample of 4500 households, 10 households were randomly selected from each of 450 collector districts (CD). A CD is a Census-defined geographical area comprising approximately 200 dwellings. CDs and households were selected using a fixed skip interval from a random starting point. An approach letter introducing the survey was sent to all selected households. Within households, the person aged 15 years or over who was last to have a birthday was selected for interview. Up to six call-backs were made to interview the selected person and there were no replacements for non-respondents. From the eligible sample (n = 4366), 808 refused to participate in the interview, 316 were not contactable after six attempts, 61 were incapacitated, 74 were unavailable for interview, 72 did not speak English, and one terminated the interview. An additional 49 households were unable to be accessed because of locked gates or because the interviewer considered it unsafe to enter. A total of 2985 interviews were completed in Spring 2004, a response rate of 68.4%. The sample size of the face-to-face survey was reduced to n = 2893 after respondents aged 15 to 17 years were excluded.

A subsample of the face-to-face survey consisting of those aged 18 years and over who owned a telephone that was listed in the EWP (n = 2206, 76.3%) was also used to compare with the telephone survey. Use of this EWP subsample enabled a closer match with the telephone sample to better compare the effect of mode rather than differences in survey sample characteristics.

Measures of SEP and health

Father and mother's highest level of education were included in both the telephone and face-to-face survey. Responses categorised as "don't know" were classed as missing. No respondents were coded as having refused to answer the questions about parents' highest level of education.

The respondent's main occupation, employment status, household income, and highest level of education were included as measures of current SEP. Respondents were asked what kind of work they had done for most of their life. Occupations were coded according to the Australian Standard Classification of Occupations (ASCO), which groups occupations according to level of education, knowledge, responsibility and on the job training and experience required [16]. Occupations were coded into five groups: 1) Manager, administrator, professional, 2) Associate professional, 3) Tradesperson, advanced clerical or service, 4) Intermediate clerical/service/sales/production/transport, and 5) Elementary clerical/sales/service/labourer. Respondents whose main occupation was home duties or student and those who had never worked or did not state their occupation were combined into a sixth category. Additional sociodemographic factors collected included age, sex, area of residence, marital status, and country of birth.

Data analysis

Data in each survey were weighted by age group, sex, geographical area, and household size to the Estimated Residential Population [17] aged 18 years and over to account for different probabilities of selection and response rates among different demographic groups, thus ensuring that the sample accurately reflected the South Australian adult population.

Univariate comparisons of the demographic profile and the distribution of the parental education variables by survey mode were made using chi-square tests in SPSS version 13 (SPSS Inc, Chicago, Illinois, USA). To examine the association between having missing data on father's or mother's highest education level and current demographic and socioeconomic variables, relative risks were calculated by log-binomial regression using Stata version 9 (StataCorp, College Station, Texas, USA). The relative risks presented were adjusted for age because increasing age was a confounding factor associated with missing data on the early life SEP variables. Relative risks were estimated using log binomial regression from the generalized linear modeling (GLM) command in Stata. This procedure was used instead of calculating odds ratios using logistic regression because odds ratios can overstate the association when the outcome is common. In this case, the dependent variable of missing data on parents' highest level of education ranged from approximately 12% to 20%. SPSS was not used here as the GLM procedure did not take account of non-integer weights. Differences were reported as significant when p < 0.01 and p < 0.001.

Results

The demographic profiles of respondents in the telephone survey, the face-to-face survey, and the EWP subsample of the face-to-face survey (those with a telephone listed in the EWP) are compared in Table 1. Compared to telephone survey respondents, those in the face-to-face survey were significantly more likely to be born outside Australia, the United Kingdom, or Ireland, have a trade, apprenticeship, certificate or diploma education level, a gross annual household income of $20,000 or less, state their employment as home duties, and have an occupation of tradesperson or advanced clerical or service. Those in the face-to-face survey were less likely than telephone respondents to have left school at 15 years of age or younger, to be unemployed or unable to work, and to have the highest status occupation. The EWP subsample of the face-to-face survey were more likely than telephone survey respondents to have a certificate or diploma education level, to state their employment as retired or home duties, and to have an occupation of tradesperson or advanced clerical or service, and less likely to be unemployed or unable to work, to have left school at 15 years of age or younger, or to have never been married.

Table 1 Demographic profile of respondents by survey mode

On questions about parents' highest level of education, respondents to the telephone survey had lower item response, and a significantly greater proportion of "don't know" responses (Table 2). The proportion of missing data for father's education ranged from 13.7% in the face-to-face survey to 19.6% in the telephone survey. Similarly, for mother's education, the proportion of missing data was 12.2% among respondents in the face-to-face survey and 20.1% among telephone survey respondents.

Table 2 Distribution of parents' highest education level by survey mode

The associations of current SEP variables with missing data on parents' highest education level were similar for the telephone and face-to-face samples and the EWP subsample. Across each survey mode, respondents with low current SEP were more likely to have missing data on father's or mother's highest education level (Table 3 and Table 4). After adjusting for age, respondents in both the telephone and face-to-face surveys who had a lower education level, lower household income, an occupation level of tradesperson or advanced clerical or service or less, or who stated their employment status as unemployed, unable to work, home duties or retired, were significantly more likely to have missing data for father's highest education level. Similarly, respondents with a lower education level, lower household income, a lower occupation level or who stated their employment status was unemployed, unable to work, or retired, were significantly more likely to have missing data for mother's highest education level. In addition, respondents in the telephone survey who were born in the United Kingdom or Ireland were also more likely to have missing data on father's and mother's highest education level.

Table 3 Unadjusted numbers, proportions and relative risks adjusted for current age for variables associated with having missing data for father's highest education level, by survey mode

Table 4 Unadjusted numbers, proportions and relative risks adjusted for current age for variables associated with having missing data for mother's highest education level, by survey mode

Discussion

Item-response to mother's and father's education level was higher in the face-to-face survey than the telephone survey. Current socioeconomic disadvantage, however, was associated with a higher prevalence of missing data on questions about parents' education in both survey modes. Socioeconomic differences in health that are based on retrospective recall of parents' education are therefore likely to be under-estimated because those who are socioeconomically disadvantaged are less likely to report their father's or mother's highest education level.

Highest education level of mother or father are commonly used indicators of early life SEP in both prospective and retrospective studies examining the association between SEP and health over the life course [15]. Previous studies relying on retrospective recall have demonstrated higher item response than obtained in this South Australian context. Only 5% of respondents in the Kuopio Ischaemic Heart Disease Risk Factor Study [18] and 6% of respondents in the Coronary Artery (Disease) Risk Development in (Young) Adults (CARDIA) study [19] had missing data for parental education questions. These studies, however, included questions about parents' education in self-administered and face-to-face surveys, and both examined younger samples than the current study. The CARDIA study asked these questions of participants aged 18 to 30 years and the Kuopio study examined middle-aged Finnish men. In addition, participants in the Kuopio study who reported that they did not know one or both of their parents were excluded from the analyses. There are no known studies that have examined the effect of mode on recall of early life SEP. The authors of the Kuopio study noted that retrospective reports of parental education are not as desirable as objective sources of data, but concluded that there was no reason why respondent recall of parents' education should be subject to recall bias [18]. The current results suggest otherwise, with respondents who were socioeconomically disadvantaged less likely to provide information about their parents' education.

The inclusion of questions about telephone ownership and listing in the EWP in the face-to-face survey allowed comparisons to be made between survey mode, independent of sampling frames. The difference in the proportion of respondents with missing data for parents' education between survey modalities could be explained in two ways. First, sociodemographic differences between the selected telephone and face-to-face samples could have influenced item response. If the telephone and face-to-face respondents differed in terms of individual characteristics that were associated with non-response to questions about parents' education, then the estimates of the proportion with missing data on these indicators may be biased by over or under-representing those with missing data [6]. There were few differences, however, between respondents in the telephone survey and those in the face-to-face survey who owned a telephone that was listed in the EWP. Furthermore, these sample characteristic variations are unlikely to have resulted in differences of the magnitude that were observed in the item non-response for parents' education.

Second, differences in interview techniques may also influence responses. For example, while there were no differences in question wording, show cards of the response categories were used in the face-to-face survey. This visual stimulus may have helped improve response rates to questions [1, 6]. The "don't know" response option was not offered to respondents in either mode. Responses were only coded as "don't know" if the respondent specifically reported it as their answer.

Face-to-face interviewers may be able to obtain a better rapport with respondents than telephone interviewers [5] and there is also often more time pressure in telephone interviews, which is likely to affect recall [6]. There may also be more opportunities during face-to-face interviews for respondents to interact with other members in the household, or to refer to documentation [6], which may have contributed to the higher item response rates for parents' education in the face-to-face survey.

There can be primacy and recency effects in category order, whereby respondents in face-to-face interviews tend to choose the first category (primacy), and telephone interview respondents choose the last category (most recent) [6]. While primacy effects could have resulted in greater proportions of respondents in the face-to-face interviews selecting the first category of parents' highest education level (completed primary school), these respondents also had a greater proportion of responses in the last category (university degree or higher).

Interview mode differences may be due to variations in interviewer effects and resulting socially desirable responses [7]. While face-to-face interviews are associated with reduced concerns about the legitimacy of interviewers, telephone interviews are associated with greater perceptions of anonymity and tend to result in less socially desirable responses [7] because the expressions and body language of the interviewer are more apparent in face-to-face interviews and may influence answers given [6]. There was no evidence in this study, however, that less socially desirable responses were gained through the anonymity of the telephone survey. In fact, the proportion who reported that their mother or father had a primary school education was higher in the face-to-face survey.

A limitation of this study is that the only indicators of early life SEP available in both the telephone and face-to-face survey were mother's and father's highest education level. Education is a valid measure of SEP [15], and using parents' education as an indicator of early life SEP would still be appropriate in prospective studies that can use more objective measures. Other indicators of early life SEP, such as parents' main occupation, or financial situation during childhood, may perform better than parents' education in terms of response and may therefore be more suitable for inclusion in surveillance systems that rely on retrospective recall. There are also factors, other than item response rates, to consider when deciding on a mode to monitor the association between SEP and health using a life course approach. These include costs and resources available to conduct such surveys, and the characteristics of the target population. Undercoverage of the target population in telephone-based sampling, for example, may be a more serious issue to consider in certain regions than item non-response.

In addition, this study does not provide information on the accuracy of recall of parents' highest level of education. While the face-to-face survey resulted in higher item-response to questions about parents' highest level of education, there is no way to compare the validity of these questions across the different survey modes.

While mode effects were observed in the proportion of respondents who answered questions about their parents' education, there were no differences between the telephone and face-to-face surveys in the demographic and socioeconomic variables associated with having missing data on the indicators of early life SEP. Respondents who were more socioeconomically disadvantaged were more likely to have missing data on mother's and father's highest education level, irrespective of survey mode.

Conclusion

The face-to-face survey resulted in higher item response to retrospective questions about parents' highest level of education than the telephone survey. This study suggests that parents' highest education level is unlikely to be the most suitable indicator of early life SEP in surveillance systems that rely on retrospective recall because of the high non-response associated with these items, regardless of the survey mode. In addition, current socioeconomic disadvantage was associated with missing data for parents' education within each survey mode. Comparing other indicators of early life SEP, such as parents' occupation, or financial situation during childhood, across different survey modes may determine indicators that are more suitable for use in surveillance systems.