- Research article
- Open Access
- Open Peer Review
Sample selection, recruitment and participation rates in health examination surveys in Europe – experience from seven national surveys
BMC Medical Research Methodology volume 15, Article number: 78 (2015)
Health examination surveys (HESs), carried out in Europe since the 1950’s, provide valuable information about the general population’s health for health monitoring, policy making, and research. Survey participation rates, important for representativeness, have been falling. International comparisons are hampered by differing exclusion criteria and definitions for non-response.
Information was collected about seven national HESs in Europe conducted in 2007–2012. These surveys can be classified into household and individual-based surveys, depending on the sampling frames used. Participation rates of randomly selected adult samples were calculated for four survey modules using standardised definitions and compared by sex, age-group, geographical areas within countries, and over time, where possible.
All surveys covered residents not just citizens; three countries excluded those in institutions. In two surveys, physical examinations and blood sample collection were conducted at the participants’ home; the others occurred at examination clinics. Recruitment processes varied considerably between surveys. Monetary incentives were used in four surveys. Initial participation rates aged 35–64 were 45 % in the Netherlands (phase II), 54 % in Germany (new and previous participants combined), 55 % in Italy, and 65 % in Finland. In Ireland, England and Scotland, household participation rates were 66 %, 66 % and 63 % respectively. Participation rates were generally higher in women and increased with age. Almost all participants attending an examination centre agreed to all modules but surveys conducted in the participants’ home had falling responses to each stage. Participation rates in most primate cities were substantially lower than the national average. Age-standardized response rates to blood pressure measurement among those aged 35–64 in Finland, Germany and England fell by 0.7-1.5 percentage points p.a. between 1998–2002 and 2010–2012. Longer trends in some countries show a more marked fall.
The coverage of the general population in these seven national HESs was good, based on the sampling frames used and the sample sizes. Pre-notification and reminders were used effectively in those with highest participation rates. Participation rates varied by age, sex, geographical area, and survey design. They have fallen in most countries; the Netherlands data shows that they can be maintained at higher levels but at much higher cost.
Sample based health examination surveys have been conducted in Europe since the late 1950’s and in the USA since 1962 . In several WHO member countries, health examination surveys are conducted under STEPS framework . These surveys have provided valuable, objective information about the health of the general population, which can be used as a basis of health monitoring  and health policy making  as well as for evaluation of health promotion activities and research.
The best value for health examination surveys is obtained when results are reliable and represent the general population well. Hospital-based surveys or other surveys targeting only on patients are limited to those receiving healthcare. They will not identify undiagnosed disease, such as cases of hypertension, or diabetes . Additionally, standardization of measurement protocols can be better controlled in specially designed health examination surveys than when data is collected from existing health care systems . Standardized health examination surveys can also provide data which is comparable across populations.
The representativeness of the health examination survey data is strongly dependent on sample selection procedures and on the participation rates achieved, and also specification, measurement and processing errors’ effect on representativeness. In questionnaire surveys, the mode of data collection may also affect participation rate . The quality of different sampling frames available varies considerably both within and between countries . Even when the best available sampling frames are used and probability samples are selected to obtain the best representation and coverage of the target population, non-participation in the survey may cause uncertainty regarding the representativeness of the survey results.
In the 1980s, it was possible to obtain a participation rate of 80 % in health examination surveys. Nowadays, participation rates of 40-50 % are common [1, 8] and this is becoming a major problem. Non-participation is selective, i.e. non-participants are more often young men, single and from lower socio-economic groups, having worse self-reported health, and they are more likely to be smokers than survey participants [9–14]. Survey non-participants also have higher total and cause specific mortality than survey participants [15–17].
In 1984–85, a response rate of 73.5 % was considered normal . Decreasing survey participation rates have occurred in the USA  and Europe [20–24]. For example, in Finland, response rates for a health interview survey fell from 84 % in 1978 to 59 % in 2002 in men and from 85 % to 71 % in women aged 25–64; the rate of decline varied considerably by age and sex .
Much research has been done on the effects of different recruitment methods on survey participation in questionnaire based surveys. A Cochrane review evaluated 47 studies (from 1954 to 2008) on the effect of pre-notification before sending the actual questionnaire for survey invitees. Mailed pre-notification increased participation in the surveys, while telephone pre-notification did not have any significant effect . Previous studies have shown that monetary incentives increase participation rates, particularly if provided unconditional on participation [25, 26].
Only a few studies have investigated the effect of recruitment methods on health examination surveys [27–29]. It could be expected that most of the findings from questionnaire based surveys would be valid for health examination surveys also. It is important to examine the effects of different recruitment methods on survey participation. We need to find the most cost-effective way to obtain as high participation rates as possible.
The reliability and generalizability of survey results depend on the data collected being representative of the population the survey is intending to describe. Sampling methods, non-response, and measurement errors are the main issues that affect survey error. While random probability sampling from an appropriate sampling frame and robust measurement protocols with adequate training and quality assurance are within the researcher’s control, non-response is more dependent on the sampled population and is of particular concern,  although it can be influenced by recruitment and promotion strategies.
A 2007 review discussed decreasing participation in epidemiological surveys . Explanations suggested included the increasing number of research studies plus the proliferation of political polls, telephone marketing, and marketing surveys that may resemble scientific surveys. Survey information that arrives in the post or by telephone may be assumed to be “junk”, arriving together with unsolicited mail or calls from commercial sources, building additional barriers to considering participation when contacted.
Health examination surveys (HESs) sampling the general population nationally have been conducted recently and on a number of previous occasions in several European countries and are being encouraged more widely . One problem of international response rate comparisons is the use of different definitions of non-response.
The aim of this paper is to describe and compare sampling and recruitment methods used in national health examination surveys in seven European countries. We evaluate the representativeness of the survey for the general population; participation rates, analysed using standardised definitions, for various measurements by population sub-groups; and changes in these over time; and the different participation rates achieved.
Survey organizers from seven European countries which had conducted at least two national health examination surveys (HESs) provided information about sampling and recruitment and details of participation rates for questionnaire or interview, specific measurements, and providing a blood sample, using standardised definitions.
Four countries ran individual-based surveys. In Finland, population-based health surveys have been conducted at five-year intervals since 1972, initially in Eastern Finland only. In 1982–1992, the surveys were part of the multinational WHO MONICA Project ; since 1992, the surveys have been called The National FINRISK Study, covering five areas in 2012  Germany has organised national health interview and examination surveys since 1984. The Studie zur Gesundheit Erwachsener in Deutschland (DEGS) [German Health Interview and Examination Survey for Adults] was conducted from November 2008 to December 2011, including both a follow-up of the 1998 survey participants and a new sample [32, 33]. In Italy, the first national health examination survey (OEC) was conducted 1998–2002 and the second survey, Observatorio Epidemiologico Cardiovascolare/Health Examination Survey (OEC/HES) in 2008-2012 . In the Netherlands, large-scale population-based health surveys were conducted annually 1987–1997. The most recent, Nederland de Maat Genomen (NLdeMaat) (Measuring the Netherlands), was carried out in two phases: Phase I in 2009 and Phase II in 2010 [35, 36]. The response rate in phase I in 2009 was only 30 % for those aged 18y and over (15 % aged 18-29y) (Fig. 1). Several changes were made, including setting the minimum age, increase of the token of appreciation, and changes in the invitation procedures, and the examination hours offered. The results from the Netherlands presented in this paper are predominantly from phase II.
Three countries held household-based surveys. The Health Survey for England (HSE), has been conducted on a new sample annually since 1991 [37, 38]. All adults (maximum 10) are eligible for inclusion. Scottish Health Surveys (SHeS) were conducted in 1995 and 1998, when only one adult per household was sampled, and in 2003 then annually from 2008 onwards, with all adults (maximum 10) included [39, 40]. For each of these, an interviewer visits the home to collect information and measure height and weight. In Scotland (up to SHeS2011) and England, a nurse visits subsequently to take further measurements (including blood pressure) and collect biological samples, including a blood sample in most years.
In the most recent Survey of Lifestyle And Nutrition (SLÁN) in Ireland in 2007, one adult per household was randomly selected for a health interview . A sub-sample of those aged under 45 years also had height, weight and waist circumference measured, while a second sub-sample of those aged over 45 years were invited to have a full health examination.
Each survey received research ethics approval from a relevant committee or institution prior to fieldwork: FINRISK 2012: Helsingin ja Uudenmaan sairaanhoitopiiri, Koordinoiva eettinen toimikunta, Helsinki (162/13/03/00/2011); DEGS 2008–2011: Ethik Kommission, Charité, Universitätsmedizin, Berlin (EA2/047/08); SLAN 2007: Royal College of Surgeons of Ireland, Dublin (RCSI REC 204 & 206); OEC/HES 2008–2012: Ethical Committee of the Istituto Superiore di Sanità, Rome (CE-ISS 08/208 and CE-ISS09-278), plus each relevant local ethics committee; NldeMaat 2009–2010: Medisch Ethische Toetsingscommissie, Universitair Medisch Centrum, Utrecht (08–420); HSE 2011: Oxfordshire REC A, National Research Ethics Service, NHS, Oxford (10/H0604/56); and SHeS 2011: Research Committee for Wales, Cardiff (08/MRE02/62). Each survey participant had provided informed consent.
Published national reports varied in how participation rates were calculated, including treatment of those who were away. Some used refusal rates; others reported broader non-response, including not contacted; contacted but not eligible (e.g. language difficulties, lack of mental capacity); and refusals. For this study, functional equivalence for participation rates was obtained as follows:
Numerators were defined as those interviewed; or those consenting to have their weight/blood pressure measured, or provide a blood sample. Technical problems with obtaining the measurement or sample and factors that would exclude these from analysis within the survey (e.g. having smoked too recently before blood pressure was measured, or the blood sample being lost) were ignored: the aim was to assess differences in willingness to participate.
For interview participation rates, the denominator was defined as all those sampled, except where the sampling frame was wrong (the address not existing or not being a private residence, for household-based surveys where the fieldstaff visited the address); the invitation letter was returned as undeliverable or ‘not known at that address’; or people died before their clinic appointment (for individual-based surveys).
For participation (consent) rates for physical and biological measurements, those capable of giving consent but ineligible for that specific measurement were excluded from the denominator (e.g. the weight of bed-bound individuals; pregnant women were not measured in England and Scotland). Other ineligible or excluded individuals, such as those who were away, in hospital or unable to give informed consent due to language problems or learning disability, remained in the denominator.
For household surveys sampling by address, in which more than one adult per household was selected (England and Scotland), interview participation rates were calculated both restricted to co-operating households (where at least one adult responded, providing household information as well as individual data), and overall, assuming the household composition of non-responding households was the same as co-operating households. For Ireland, the sampling frame for the examination survey was a sub-sample of those aged 45 years or older who were interviewed in the household survey, so participation rates have been multiplied by the household participation rate.
The four participation rate outcomes were: consent rate for blood pressure measurement (primary outcome measure), participation in the interview, and consent for weight measurement and for providing a blood sample. These represented four stages of the surveys. Using the definitions above, we determined participation rates by age and sex, geography, and year. Depending on the data available, national participation rates were compared with data from the capital city (Greater London, Rome), the primate cityFootnote 1 (Glasgow City/Greater Glasgow area),  or major metropolitan areas (Berlin, Munich, Hamburg and Frankfurt combined, Helsinki-Vantaa).
Where possible, age-standardized rates using the European Standard Population  were calculated for participants aged 35–64 years, covering the age-groups included in most surveys. Due to differences in the age-groups included and the calculations of participation rates between individual-based and household-based surveys, results presented are not directly comparable between countries.
Sampling in European national health examination surveys
Sampling frames in these seven national HESs can be divided into two groups; individual-based and household-based. All individual-based sampling frames were population registers, either national or local. Population registers, commonly available in Europe, list people living in the country/region but obviously there are some technical problems with homeless individuals, etc. They may be listed in the population register but without a permanent address. In three surveys, household-based -sampling frames were used because of the lack of population registers in those countries: in these surveys, the sampling frames were address files (Table 1).
All surveys covered both men and women. In England and Scotland, all people (without age limit) were included in the survey. In Ireland, people from age of 18 years without an upper age limit were included; those aged 18–44 years had their height, weight and waist circumference measured and those 45+ years old were invited for a full physical examination. The other four surveys had narrower age limits. The age group 35–64 years was common to all seven surveys (Table 1).
In each survey, all people living in the country were included in the target population to the extent it was possible to list them in the sampling frames, i.e. the target population was residents not citizens. For practical reasons, in the three household surveys, only people living in private households were included, i.e. institutionalized people were excluded from the target population (Table 1).
In each survey, probability sampling was used. In three surveys (Finland, Italy and the Netherlands), survey areas were selected based on availability/feasibility criteria and individuals within each selected areas were selected using random sampling from the population register. In these surveys, samples were stratified by age group and sex. In Germany, where part of the sample was based on re-invitation of participants of the previous HES (conducted in 1998), the additional new sample was selected using a two stage random sample . In the three surveys using household sampling, two stage sampling was used. In the first stage, the local areas were selected (postcode sectors). In the second stage, a set number of households were selected from each of the selected areas. Within selected households, sampling procedures to select individuals varied between surveys (Table 1).
The original sample size in these surveys varied from 10,000 to 17,117 individuals in surveys based on samples of individuals and from 8,736 to 19,185 households in surveys based on household sampling (Table 1).
In individual-based surveys, people who had died or moved out from the survey area were considered as ineligible for the survey. In Germany, people who did not understand basic German and in Italy people who were outside the survey area for the entire survey period were also classified as ineligible for the survey. In household-based surveys, vacant, demolished, and non-residential buildings were classified as ineligible (Table 1).
Eligible sample sizes (original sample size minus those not eligible) for individual based surveys varied from 9,905 to 16,477 individuals. For household surveys, eligible sample sizes were from 7,564 to 16,681 households. In surveys using population registers as sampling frames, the proportion who were ineligible was 1 %-7 % and in surveys using address lists as sampling frames, 10 %-13 % (Table 1).
In all surveys except in Finland and Ireland, the first contact with survey invitees was either an advance letter or an invitation letter. In Finland, survey invitees received first a short message service (SMS) text message which was then shortly followed by an invitation letter. In Ireland, the invitation to take part in the health examination was given by the interviewer who located the address (Table 2).
The recruitment processes after the initial contact with the survey invitees varied between countries. There were different combinations of reminder letters, phone calls and home visits. The number of contact attempts varied from one upwards (Table 2).
In all surveys, feedback about the examination results was provided for survey participants. In Italy lifestyle advice was also provided for survey participants. In England, Germany, the Netherlands and Scotland, monetary incentives were used to promote participation. In Germany, cash was provided at the end of the health examination, while in other surveys, vouchers were given. In the Netherlands, receiving the voucher was conditional on participation while in England and Scotland, the voucher was sent to the household with the advance letter, i.e. it was unconditional on participation (Table 2).
In England, Ireland and Scotland, face-to-face interviews and self-completed questionnaires were filled in during the interview at the survey participant’s home. In Italy, an interview was conducted at the examination centre. In Finland, Germany and the Netherlands, questionnaires were sent to the survey invitees and they were filled in before coming to the examination site. The questionnaires were checked by survey staff during the examination visit. In Germany, additional information was collected by a second self-administered questionnaire and personal interviews at the examination site (Table 2).
In England and Scotland, health examinations were carried out by trained nurses in the survey participant’s home; additional questions were asked by the nurse during this visit. In other countries, health examinations were conducted in fixed examination centres (Table 2).
In Finland, health examinations were carried out during working days (Monday-Friday) while in other countries examination times were also offered at the weekends (Saturday and/or Sunday) (Table 2).
The times of the day when health examinations were carried out also varied between countries. In England and Scotland, which conducted the health examinations at the survey participant’s home, times throughout the day from morning to late evening were offered. In Ireland, appointments were available all day. The time frame for the examinations was narrowest in Finland: from 11:00 until 18:00. In Germany and Italy both morning and evening hours were available. In the Netherlands, the survey started (phase I) with only morning hours but to boost participation, evening hours were offered in phase II of the survey (Table 2).
The common age group in all surveys was 35–64 years, except in Ireland with no blood pressure measurements or sample collection for participants aged 35–44. Participation rates were generally higher in women than in men, except in Ireland, where they were equal, and in Germany, where men aged 35–64 had a higher participation rate. Participation rates increased with age up to 64 in all surveys. For older age groups, participation rate was lower in women in Germany and in both sexes in Italy (Table 3).
In surveys where all physical measurements and sample collection were done at examination centres, participation rates differed little between weight measurement, blood pressure measurement and blood sample collection, varying by only 0–3 percentage points (pp) in Italy, Ireland and the Netherlands; in Finland, blood sample consent rates were 5–8 pp lower than interview rates; in Germany, blood sample consent rates were 4–9 pp lower than the BP measurements. In Germany, some of the difference was due to people who were unwilling or unable to attend the centre having a telephone interview.
However, in England and Scotland, the fully household-based surveys where two visits by survey staff were required and measurements were conducted in the participants’ home, there was a marked reduction in participation rate for each successive measure in the survey of 38–46 pp among adults in participating households (Table 3).
Except for Finland, participation rates for capital or primate cities and major metropolitan areas were consistently around 8–15 pp lower than the national average (Table 4). In England and Scotland, differences occurred in both the initial household participation rate and the proportion having a nurse visit but individual participation rates within co-operating households varied little from the national average.
Participation rates have fallen over the past decade in each country for which comparable data were available, apart from the Netherlands NLdeMaat Phase II (Table 5), with its substantial improvement in participation rates compared with phase I (Fig. 1).
The average annual change in the other countries ranged from −0.7 to −1.5 pp; they were larger in women than men in Finland and Germany. In England, the falling participation rates for blood pressure measurement were greater in men than women and in younger than older people (Fig. 2). Falling participation rates in England occurred primarily for household participation (Fig. 3a), but even within co-operating households, participation in successive stages each fell over time to a greater extent (Fig. 3b and c). The reduction in initial participation rates over time in Finland (Fig. 4) is similar to the household and estimated interview participation rates in England over the same period.
Participation rates in the four countries using monetary incentives (England, Germany, the Netherlands and Scotland) were lower than in the other surveys. In Finland, which had the highest participation rate among individual-based surveys, pre-notification and a reminder before appointment time by short message service (SMS) was used. No clear correlation was observed between the range of examination days and hours of day offered and the participation rates obtained.
The main strength of this study is the inclusion of several European countries with significant experience of conducting nationally-representative health examination surveys. Differing definitions of response rates is a well-recognised issue [44, 45]. The limitation of these different definitions of non-response, in each individual survey, was overcome by use of agreed uniform definitions for both numerators and denominators. As with comparisons of the findings of disparate heath surveys across Europe,  the lack of uniformity of survey methods prevents assessment of the impact of cultural differences within countries, and aspects of sampling, recruitment, and data collection on participation rates. Analysis is therefore limited primarily to comparisons within each country.
In countries which have either national or local population registers, these were used as the sampling frames for individuals. Countries without population registers constructed their sample using address files. Population registers have good coverage of the population; updating procedures are usually regulated by law  and occur regularly. Population registers include information about the name of the person, their address and often their sex and age as well. Address files do not list individuals but buildings; they often include vacant buildings, offices and other non-domestic houses. The updating frequency of address files varies between countries. For example in Ireland, the GeoDirectory identified 2.7 % of residential addresses as vacant while the Census reported 15 % of residential addresses as vacant . The large proportion of ineligible addresses in address files (>10 % on average) increases the cost of making contacts as interviewers have to check all addresses.
For practical reasons, people living in institutions are often excluded from health examination surveys, for example in the three surveys using household-based sampling. When people are asked to come to the fixed examination clinic, it may be impossible for institutionalized persons to participate. Also in cases where home visits are conducted routinely, or offered as an alternative for those not able to come to the examination site, inclusion of institutionalized persons may be difficult. Many institutionalized persons have illnesses which prevent their participation or ability to provide informed consent; in the case of prisoners, both the safety of the survey staff and the privacy of the examination may be difficult to guarantee. For the working aged population (18–64 years), the number of institutionalized people is a small proportion of the population and should not affect the representativeness of the survey results. In older age groups (65+ years), the proportion of the population living in institutions (such as care homes) is larger.3 %-6 % of people aged 64 and over in OECD countries, are living in long-term care institutions : excluding them may affect the representativeness of the results.
The recent European recommendations (European Health Examination Survey) for conducting a national health examination survey determined that a sample size of at least 4000 persons is needed to survey the 25–64 year age group . The EHES recommendations are similar to STEPS recommendations . The sample size should be made up of equal numbers of men and women in each 10-year age group (500 in each group) in order to obtain nationally representative results. In each of the seven national health examination surveys, the sample size fulfilled this criterion even though in most of the surveys, a wider age group was covered.
Sending a short pre-notification letter for survey invitees before mailing out the invitation and questionnaire is technically easy to organize but for large population surveys it increases the cost of the survey. In some countries, sending a personalized pre-notification is not possible due to the sampling methods. In Finland, it was possible to send SMS messages for survey invitees as mobile phone numbers are widely available. In a recent, local HES within Finland, those receiving an SMS reminder were up to 25 pp more likely to participate , although telephone reminders have not been found to have a significant effect . The rapid increase of mobile phone subscriptions and at the same time decline in land-line subscriptions is opening new possibilities for contacting survey invitees  in countries where listings of mobile phone numbers are available.
In health examination surveys, home visits have also been used to contact survey invitees who do not respond to the invitation . In the Netherlands, this was an effective way to convince people to participate, although it was instituted at the same time as several other changes to the recruitment procedures, making formal evaluation difficult.
For health examination surveys, a Norwegian study reported that many participants felt that incentives would impose commercial features on the survey and could undermine the confidence in the survey. They also noted that there were clear differences between population groups regarding acceptance of incentives. People from lower socio-economic classes and younger age groups, the population groups which tend to have the lowest participation rates in health examination surveys, had more positive attitude towards incentives .
Although the larger the monetary incentive, the bigger the effect on participation rate , there are diminishing effects of greater amounts. In some countries, research ethics committees have prevented the use of monetary incentives in health examination surveys. Therefore, not all surveys have had an equal opportunity to use monetary incentives. In England, use of an unconditional £5 voucher in the advance letter to the household increased household participation rates by 3 % . In large health examination surveys, providing monetary incentives can have a significant effect on the survey budget but it may also reduce number of contact attempts needed and therefore reduce survey staff costs.
Little has been reported about people’s preferences for day of the week and time of the day when it would be most convenient for them to participate in the survey. One would assume that availability depends on people’s daytime commitments and lifestage. It has been shown in the Health Survey for England that people attending the survey outside ‘office hours’ have different demographic and socio-economic profiles than those attending during the day on weekdays . Therefore, it is important to organize health examination surveys so that everyone is able to be seen when convenient for the participant. This means offering a wide range of opening hours for examination clinics and/or home visits to maximize participation. On the other hand, extensive opening hours makes it difficult to organize collection of fasting blood samples (at least when 8–12 hours fasting is required), which are usually collected in mornings after overnight fasting. Longer data collection hours can also require more field work personnel, which increase the survey cost.
Health examination survey participation rates across seven European countries were generally higher in women and older people, whether conducted in participants’ own homes or an examination centre. Participation rates in the National Health and Nutrition Survey (NHANES) in the USA were also higher among women. Although participation rates in NHANES increased by age among women until recently, the declining participation with age among older men now also occurs in women .
Participation rates declined over time by 7–15 pp in the past decade in the four countries for which comparable data were available. In NHANES, reported (interview and) examination response rates fell from (99 % and) 74 % of adults aged 25-74y in 1971–74 to (70 % and) 67 % aged 30-69yFootnote 2 in 2011-2012 . These figures are not directly comparable with rates in our study as the NHANES data may not match our standardised definitions, however, the NHANES response rates remain high. It is possible that financial barriers to routine healthcare in the USA mean that the opportunity for a free health check has been a greater incentive in those countries than in European countries with universal healthcare.
Participation rates for England and Scotland presented in the Tables are based on households in which at least one adult responded, as the age and number of household members is otherwise unknown. After multiplying individual rates by household response rates, interview rates in England and Scotland were similar to or better than in other countries. However, the two-stage method used in England and Scotland for data collection had a marked reduction in participation in the second stage, with blood pressure measured in substantially fewer individuals than weight. There was further reduction in participation rates when blood sample consent was compared with blood pressure measurements, which occurred in the same survey nurse visit. This may reflect an unwillingness to provide blood samples in a domestic, non-clinical environment, particularly when the samples are not required for clinical reasons.
Participation rates in the examination elements in the five countries where participants attend a central location were uniformly high as a proportion of those interviewed, and participation rates for the three measurements varied little. However, the proportion attending as a percentage of the whole sample varied widely by age, sex, and country. In the Netherlands, the first phase of NLdeMaat had much lower participation rates, although already using more intensive recruitment than in the 1993–1997 MORGEN study. Phase II of NLdeMaat demonstrated that it was feasible to reproduce 'historical' participation rates, but at a much higher cost than previously.
The lower participation rate in capital cities has been noted before within individual studies ; we have confirmed this as a problem in most primate cities and metropolitan areas. Lower contact rates (a necessary first step before participation can be requested in surveys based on an interviewer calling in person, such as the household-based surveys) occur in more highly urbanized areas . Groves and Couper relate this in part to these areas have a higher preponderance of less accessible household types, with more multi-unit dwellings, single-person households (thus a reduced likelihood of anyone being home when an interviewer calls), renter-occupied homes, more time spent commuting, and more entertainment options . Large metropolitan areas also have a younger and more mobile population (increasing non-contact rates), greater poverty (also increasing non-contact ), and larger immigrant populations with more language barriers to survey participation, both through comprehension problems [33, 58] and language-related barriers of perceived relevance . In the German DEGS1 study reported here, participation rates were inversely proportion to conurbation size (53 % in localities with a population <5,000; 35 % in cities of 500,000+) . Participation rates were 10 pp higher among German than other nationals in DEGS1 .
The main concern over lower participation rates is the extent to which the results are representative of the underlying population, with non-response bias if participation is not random. Groves summarised five theoretical models relating response propensity to the likelihood of non-response bias . He concluded that while non-response bias does occur, it is not a simple function of the non-response rate: higher response rates reduce the risk of such bias but not the actual presence of such bias. Non-respondents to health examination surveys have higher mortality than participants [11, 15–17]. Socio-economic position is associated with non-response [11, 23]; one study found that deprivation was associated with difficulty in making contact but individuals in less deprived areas had higher refusal rates ), Other studies found higher response rates among more educated individuals . Some but not all studies have found both demographic and health differences between those responding early to a survey and those requiring greater effort to contact or to encourage into the survey [61, 62]. Representativity indicators have been proposed to replace response rates; they can be generated during fieldwork to target less represented potential participants (a responsive survey design) . Field substitution to address differential non-response has little effect on the results  but post-survey adjustments (e.g. non-response; weighting) removes these differences, [61, 64, 65] with a suggestion to reduce fieldwork costs (or increase sample size) by reducing efforts to contact or convert non-responders and weighting data instead [61, 66].
Factors influencing decisions to participate in a survey include societal and individual level factors, survey design, and interviewer attributes . In addition, the perceived relevance or sensitivity of the topic can affect participation. Thus health surveys may have health-related non-response bias both because those whose health is less good are more likely to be available  to contact, and because the survey may appear more salient to them. Gibson et al. have found reporting heterogeneity for similar health questions on limiting longterm illness, with lower prevalence among census respondents than health survey participants . In contrast, alcohol consumption is often under-estimated by surveys; Gray et al. compared morbidity-linked survey and population data and proposed a method to develop more accurate (higher) alcohol consumption estimates using multiple imputation . However, Hall et al. found that non-response weighting bias addressed health variables adequately, other than smoking, but not attitudinal variables .
Seven national health examination surveys conducted in Europe in 2007–2012 can be classified either as individual-based or as household-based surveys. For each survey, the best available sampling frame within the country was used and recruitment methods were adjusted for national needs. Participation rates differ by age, sex, country, and survey design but have fallen over time in most countries. There is no single correct way to organize a national HES but national circumstances, cultural requirements and norms, and the resources available have important roles in effective survey organization. Experience in the Netherlands shows that higher participation rates can be maintained through intensive efforts but these incur considerable costs.
The primate city is a major city that is at least twice as large as the next largest city and more than twice as significant, e.g. Greater Glasgow in Scotland.
Data are not published for the same age groups in earlier and more recent years. In 2011–12, response rates were substantially higher among those aged 30-69y than those 20-29y or 70+, so the age-groups presented in the text underestimate the fall in response rates.
Grotvedt L, Kuulasmaa K, Tolonen H, Heldal J, Graff-Iversen S. 7. Sampling and recruitment. In: Tolonen H, Koponen P, Aromaa A, Conti S, Graff-Iversen S, Grotvedt L, Kanieff M, Mindell J, Natunen S, Primatesta P, et al., editors. Review of Health Examination Surveys in Europe. Helsinki: Publications of the National Public Health Institute B18/2008; 2008. Available at https://www.julkari.fi/bitstream/handle/10024/103057/2008b18.pdf?sequence=1 (Accessed 07/09/2015).
STEPwise approach to surveillance (STEPS) http://www.who.int/chp/steps/en/ (Accessed 07/09/2015)
Borodulin K, Vartiainen E, Peltonen M, et al. Fourty-year trends in cardiovascular risk factors in Finland. Eur J Public Health. 2014.
Oyebode O, Mindell J. Use of data from the Health Survey for England in obesity policy making and monitoring. Obesity Reviews. 2013;14(6):463–76.
Tolonen H, Koponen P, Mindell JS, Mannisto S, Giampaoli S, Dias CM, et al. Under-estimation of obesity, hypertension and high cholesterol by self-reported data: Comparison of self-reported information and objective measures from health examination surveys. Eur J Public Health. 2014, doi:10.1093/eurpub/cku074.
Tolonen H, Koponen P, Mindell J, Mannisto S, Kuulasmaa K. European Health Examination Survey--towards a sustainable monitoring system. Eur J Public Health. 2014;24(2):338–44.
Biemer PP. Measurement errors in surveys. Hoboken, N.J.: Wiley-Interscience; 2004.
Blackman S, Goldstein KM. Some aspects of a theory of community mental health. Community Ment Health J. 1968;4(1):85–90.
Boshuizen HC, Viet AL, Picavet HS, Botterweck A, van Loon AJ. Non-response in a survey of cardiovascular risk factors in the Dutch population: determinants and resulting biases. Public Health. 2006;120(4):297–308.
Drivsholm T, Eplov LF, Davidsen M, Jorgensen T, Ibsen H, Hollnagel H, et al. Representativeness in population-based studies: a detailed description of non-response in a Danish cohort study. Scand J Public Health. 2006;34(6):623–31.
Harald K, Salomaa V, Jousilahti P, Koskinen S, Vartiainen E. Non-participation and mortality in different socioeconomic groups: the FINRISK population surveys in 1972–92. J Epidemiol Community Health. 2007;61(5):449–54.
Sogaard AJ, Selmer R, Bjertness E, Thelle D. The Oslo Health Study: The impact of self-selection in a large, population-based survey. Int J Equity Health. 2004;3(1):3.
Tolonen H, Dobson A, Kulathinal S. Effect on trend estimates of the difference between survey respondents and non-respondents: results from 27 populations in the WHO MONICA Project. Eur J Epidemiol. 2005;20(11):887–98.
Van Loon AJ, Tijhuis M, Picavet HS, Surtees PG, Ormel J. Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol. 2003;13(2):105–10.
Hara M, Sasaki S, Sobue T, Yamamoto S, Tsugane S. Comparison of cause-specific mortality between respondents and nonrespondents in a population-based prospective study: ten-year follow-up of JPHC Study Cohort I. Japan Public Health Center. J Clin Epidemiol. 2002;55(2):150–6.
Jousilahti P, Salomaa V, Kuulasmaa K, Niemela M, Vartiainen E. Total and cause specific mortality among participants and non-participants of population based health surveys: a comprehensive follow up of 54 372 Finnish men and women. J Epidemiol Community Health. 2005;59(4):310–5.
Une H, Miyazaki M, Momose Y. Comparison of mortality between respondents and non-respondents in a mail survey. J Epidemiol. 2000;10(3):136–9.
Boniface DRE, Tefft ME. Dietary fats and 16-year coronary heart disease mortality in a cohort of men and women in Great Britain. Eur J Clin Nutr. 2002;56:786–92.
Atrostic BK, Bates N, Burt G, Silberstein A. Nonresponse in U.S. government household surveys: consistent measures, recent trends, and new insights. J Off Stat. 2001;17(2):209–26.
De Heer W. International reponse trends. Results of an international survey. J Off Stat. 1999;15(2):129–42.
de Leeuw E, de Heer W. Trends in Household Survey Nonresponse: A Longitudinal and International Comparison. In: Groves R, Dillman D, Eltinge J, Little RJ, editors. Survey Nonresponse. New York: Wiley; 2002. p. 41–54.
Aromaa A, Koponen P, Tafforeau J, Vermneire C, Primatesta P, Marmot M. Health surveys: evaluation and recommendations. Subproject reports of Phase 2 of the project Health surveys in the EU: HIS and HIS/HES Evaluations and Models. Helsinki: National Public Health Institute; 2003.
Tolonen H, Helakorpi S, Talala K, Helasoja V, Martelin T, Prattala R. 25-year trends and socio-demographic differences in response rates: Finnish adult health behaviour survey. Eur J Epidemiol. 2006;21(6):409–15.
Williams B, Entwistle V, Haddow G, Wells M. Promoting research participation: Why not advertise altruism? Soc Sci Med. 2008;66(7):1451–6.
Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;3, MR000008.
Church AH. Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opin Quartely. 1993;57:62–79.
Antonsen S. Motivasjon for deltakelse i helseundersokelser. Norsk Epidemiologi. 2005;15(1):99–109.
Holle R, Hochadel M, Reitmeir P, Meisinger C, Wichmann HE. Prolonged recruitment efforts in health surveys: effects on response, costs, and potential bias. Epidemiology. 2006;17(6):639–43.
Mindell J, Aresu M, Becares L, Tolonen H. Representativeness of participants in a cross-sectional health survey by time of day and day of week of data collection. Eur J Public Health. 2012;22(3):364–9.
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.
Tunstall-Pedoe H, Kuulasmaa K, Tolonen H, Davidson M, Mendis S, for The WHOMP, et al. MONICA Monograph and Multimedia Sourcebook. Geneva: WHO; 2003.
Gosswald A, Lange M, Dolle R, Holling H. The first wave of the German Health Interview and Examination Survey for Adults (DEGS1): participant recruitment, fieldwork, and quality management. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2013;56(5–6):611–9.
Kamtsiuris P, Lange M, Hoffmann R, Schaffrath Rosario A, Dahm S, Kuhnert R, et al. The first wave of the German Health Interview and Examination Survey for Adults (DEGS1): sample design, response, weighting and representativeness. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2013;56(5–6):620–30.
Giampaoli S, Vanuzzo D, Survey edGdRdPOECHE. La salute cardiovascolare degli italiani. G Ital Cardiol. 2014;15(1):55.
Nederland de Maat Genomen http://www.rivm.nl/Onderwerpen/Onderwerpen/N/Nederland_de_Maat_Genomen (Accessed 07/09/2015)
Blokstra A, Vissink P, Venmans LAMJ, Holleman P, van der Schouw YT, Smit HA, et al. Nederland de Maat Genomen, 2009–2010. Monitorin van risicofactoren in de algemene bevolking. In: RIVM Rapport 260152001/2011. 2011.
Craig R, Mindell J, editors. Health Survey for England 2011. Volume 2. Methods and documentation. London: The Health and Social Care Information Centre; 2012.
Mindell J, Biddulph JP, Hirani V, Stamatakis E, Craig R, Nunn S, et al. Cohort Profile: The Health Survey for England. Int J Epidemiol. 2012;41(6):1585–93.
Gray L, Batty GD, Craig P, Stewart C, Whyte B, Finlayson A, et al. Cohort Profile: The Scottish Health Surveys Cohort: linkage of study participants to routinely collected records for mortality, hospital discharge, cancer and offspring birth characteristics in three nationwide studies. Int J Epidemiol. 2010;39(2):345–50.
Rutherford L, Sharp C, Bromley C, editors. The Scottish Health Survey 2011. Volume 3: Technical Report. Edinburgh: The Scottish Government; 2012.
Morgan K, McGee H, Watson D, Perry I, Barry M, Shelley E, et al. SLÁN 2007: Survey of Lifestyle, Attitudes & Nutrition in Ireland. Main Report. Dublin: Department of Health and Children; 2008.
Jefferson M. The law of the primate city. Geogr Rev. 1939;29:226–32.
Ahmad OB, Boschi-Pinto C, Lopez AD, Murray CJL, Lozano R, Inoue M. Age standardization of rates: a new WHO standard. In: Organization WH, editor. GPE Discussion Paper Series: No 31 EIPH/GPE/EBD. 2001. Availalbe at http://www.who.int/healthinfo/paper31.pdf (Accessed 07/09/2015).
Wiseman F, Billington M. Comment on a Standard Definition of Response Rates. J Mark Res. 1984;21:336–8.
Lynn P, Beerten R, Laiho J, Martin J. Recommended Standard Final Outcome Categories and Standard Definitions of Response Rate for Social Surveys. In: Working Papers of the Institute for Social and Economic Research. Colchester: ISER, University of Essex; 2001.
Gray L, Merlo J, Mindell J, Hallqvist J, Tafforeau J, O’Reilly D, et al. International differences in self-reported health measures in 33 major metropolitan areas in Europe. Eur J Public Health. 2012;22(1):40–7.
Tolonen H. Towards the high quality of population health surveys. Standardization and quality control. Kuopio, Finland: University of Finland; 2005. Available at http://urn.fi/URN:ISBN:951-740-546-4 (Accessed 07/09/2015).
OECD. Society at a Glance 2006. OECD Social Indicators. In.: OECD Publishing. doi: 10.1787/soc_glance-2006-en; 2007.
Heldal J, Jentoft S. 2. Target population and sample size. In: Helsinki TH, editors. EHES Manual Part A Planning and preparation of the survey. Volume Directions 2013_001. National Institute for Health and Welfare; 2013. Available at http://urn.fi/URN:ISBN:978-952-245-842-1. (Accessed 07/09/2015)
World Health Organization. STEPS Manual. Geneva: World Health Organization; 2014. Available at http://www.who.int/chp/steps/manual/en/index2.html. (Accessed 07/09/2015).
Tolonen H, Aistrich A, Borodulin K. Increasing health examination survey participation rates by SMS reminders and flexible examination times. Scand J Public Health. 2014;42(7):712–7.
World Telecommunication/ICT Indicators database 2013 (17th Edition) [http://www.itu.int/en/ITU-D/Statistics/Pages/publications/wtid.aspx]. (Accessed 07/09/2015).
Craig R, Mindell J, Hirani V, editors. Health Survey for England 2008. Volume 2. Methods and documentation. London: The Information Centre for Health and Social Care; 2009.
NHANES response rates [www.cdc.gov/nchs/nhanes/response_rates_cps.htm]. (Accessed 07/09/2015).
Mindell JS, Tipping S, Pickering K, Hope S, Roth MA, Erens B. The effect of survey method on survey participation: Analysis of data from the Health Survey for England 2006 and the Boost Survey for London. BMC Med Res Methodol. 2010;10(1):83.
Groves RM, Couper MP. Nonresponse in household interview surveys. New York: Wiley; 1998.
Goodman A, Gatward R. Who are we missing? Area deprivation and survey participation. Eur J Epidemiol. 2008;23(6):379–87.
Ahlmark N, Algren M, Holmberg T, Norredam M, Nielsen S, Blom A, et al. Survey nonresponse among ethnic minorities in a national health survey - a mixed-method study of participation, barriers, and potentials. Ethn Health. 2015;20(6):611–32.
Pelzl I, Pohlabeln H, Relneke A, Ahrens W. Externe Qualitatssicherung der ersten Welte der Studie zur Gesundheit Erwachsener in Deutschland (DEGS1). [External quality assuranceof the first wave of the German Health Interview and Examination Survey for Adults (DEGS1)]. Bundesgesund Heitsbl. 2013;56:637–42.
Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Quartely. 2006;70(5):646–75.
Davern M, McAlpine D, Beebe TJ, Ziegenfuss J, Rockwood T, Call KT. Are Lower Response Rates Hazardous to Your Health Survey? An Analysis of Three State Telephone Health Surveys. Health Serv Res. 2010;45:1324–44.
Korkeila K, Suominen S, Ahvenainen J, Ojanlatva A, Rautava P, Helenius H, et al. Non-response and related factors in a nation-wide health survey. Eur J Epidemiol. 2001;17(11):991–9.
Shlomo N, Skinner C, Schouten B. Estimation of an indicator of the representativeness of survey response. J Stat Planning Infer. 2012;142:201–11.
Van der Heyden J, Demarest S, Van Herck K, De Bacquer D, Tafforeau J, Van Oyen H. Association between variables used in the field substitution and post-stratification adjustment in the Belgian health interview survey and non-response. Int J Public Health. 2014;59(1):197–206.
Forthofer RN. Investigation of nonresponse bias in NHANES II. Am J Epidemiol. 1983;117(4):507–15.
Hall J, Brown V, Nicolaas G, Lynn P. Extended field efforts to reduce the risk of non-response bias: Have the effects changed over time? Can weighting achieve the same effects? Bull Méthode Social. 2013;117:5–25.
Groves RM, Cialdini RB, Couper MP. Understanding the decision to participate in a survey. Public Opin Q. 1992;56:475–95.
Gibson A, Hewson P, Asthana S. Modelling the nature, scale and consequence of health-related non-response bias in Health Survey for England data. In: Health Survey Users Meeting. London: UK Data Service; 2013.
Gray L, McCartney G, White IR, et al. Use of record-linkage to handle non-response and improve alcohol consumption estimates in health survey data: a study protocol. BMJ Open. 2013;3:e002647.
Vartiainen E, Borodulin K, Sundvall J, Laatikainen T, Peltonen M, Harald K, et al. FINRISKI Tutkimus: Väestön kolesterolitaso on vuosikymmenien laskun jälkeen kääntynyt nousuun. Suomen Lääkärilehti (Finnish Medical Journal). 2012;35:2364–8.
We thank the members of the HES Response Rates Working Group for assistance with revising the analysis plan, providing information and results from their own countries, discussing the findings, and commenting on the draft manuscript. We also thank the researchers, office staff, fieldwork staff, participants in and funders of each of the surveys;; and researchers in the EHES Reference Centre for bringing us together.
The work of Hanna Tolonen was supported by Academy of Finland grant no 266251.
This project arose from work undertaken as part of the European Health Examination Survey (EHES) project. The EHES Pilot Project received funding from the European Union/DG Sanco (Agreement number 2009 23 01 and Service Contract SANCO/2008/C2-SI2.53818 EHES). The work of HT was partially funded by Academy of Finland (266251); SM was supported by the Academy of Finland (136895 and 263836), WV was funded by the Dutch Ministry of Health and Welfare. The funders of the surveys played no part in this work, nor in decisions to publish it.
The views expressed here are those of the authors and they do not represent the Commission’s official position.
HES response rate working group
Charlotte Mann, Jennifer Mindell, Nicola Shelton (England); Satu Männistö, Hanna Tolonen (Finland); Antje Gößwald, Panagiotis Kamtsiuris (Germany); Karen Morgan (Ireland); Chiara Donfrancesco, Simona Giampaoli; Cinzia Lo Noce, Luigi Palmieri (Italy); Monique Verschuren (Netherlands); Catherine Bromley, Lisa Rutherford (Scotland).
The authors declare that they have no conflict of interest.
JM conceived the idea, convened the group, proposed the analyses, and wrote the first draft of the paper. All authors (JM, SG, AG, PK, CM, SM, KM, NJS, WMMV, and HT) meet the three criteria for authorship, namely they discussed and revised the analysis plan, conducted the main analyses for this paper of their own countries’ survey data, revised the drafts critically for important intellectual content, and read and approved the final manuscript.
Availability of data and materials
About this article
Cite this article
Mindell, J.S., Giampaoli, S., Goesswald, A. et al. Sample selection, recruitment and participation rates in health examination surveys in Europe – experience from seven national surveys. BMC Med Res Methodol 15, 78 (2015). https://doi.org/10.1186/s12874-015-0072-4
- Health examination survey
- Sampling frame
- Sample size
- Participation rates