Who does not participate in a follow-up postal study? a survey of infertile couples treated by in vitro fertilization
BMC Medical Research Methodology volume 12, Article number: 104 (2012)
A good response rate has been considered as a proof of a study’s quality. Decreasing participation and its potential impact on the internal validity of the study are of growing interest. Our objective was to assess factors associated with contact and response to a postal survey in a epidemiological study of the long-term outcome of IVF couples.
The DAIFI study is a retrospective cohort including 6,507 couples who began an IVF program in 2000-2002 in one of the eight participating French IVF centers. Medical data on all 6,507 couples were obtained from IVF center databases, and information on long-term outcome was available only for participants in the postal survey (n = 2,321). Logistic regressions were used to assess firstly factors associated with contact and secondly factors associated with response to the postal questionnaire among contacted couples.
Sixty-two percent of the 6,507 couples were contacted and 58% of these responded to the postal questionnaire. Contacted couples were more likely to have had a child during IVF treatment than non-contactable couples, and the same was true of respondents compared with non-respondents. Demographic and medical characteristics were both associated with probability of contact and probability of response. After adjustment, having a live birth during IVF treatment remained associated with both probabilities, and more strongly with probability of response. Having a child during IVF treatment was a major factor impacting on participation rate.
Non-response as well as non-contact were linked to the outcome of interest, i.e. long-term parenthood success of infertile couples. Our study illustrates that an a priori hypothesis may be too simplistic and may underestimate potential bias. In the context of growing use of analytical methods that take attrition into account (such as multiple imputation), we need to better understand the mechanisms that underlie attrition in order to choose the most appropriate method.
Participation rates in cohort studies have decreased during the last two decades . A good response rate has been considered as a proof of a study’s quality . Therefore, decreasing participation and its potential impact on the internal validity of studies are of growing interest [3–5]. Decreasing participation raises the question of “how much is too much?”, but above all, the question of “who do you lose?” [6–8].
In a substantial number of published studies, information about participation rates is not given or is incomplete, especially in cohort studies . This underreporting may be in part linked to the epidemiological tendency to consider low participation rate as a sign of inferior quality . Moreover, participation rates may be overestimated, since authors may define participation without taking into account all the different steps along the path of data collection [1, 9]. In studies that did report participation rates, loss to follow-up was sometimes surprisingly high: the French COCON study on women’s contraception practices reported a loss of one-third of its members between the first and third waves, a period of 2 years . In the UK Millennium Cohort Study, 20% of parents who participated in the first sweep did not respond to the second sweep two years later [11, 12]. Participation rates lower than 50% have been reported in various follow-up studies such as the Danish National Birth Cohort [13, 14], the Australian 45 and Up Study [15, 16], the Australian Longitudinal Study on Women’s Health  and the French GAZEL cohort [18, 19].
Participation rates may even be lower in specific populations such as infertile couples receiving or having received medical treatment, as they could be reluctant to participate after the end of their treatment in a study that reminds them of a physically and psychologically exhausting experience [20–23]. Because of these high non-participation rates, selection bias could be a serious issue in such studies. Research on factors associated with non-participation is quite rare because very often there is no information at all on those who did not participate [1, 5, 8, 24]. However, if we are to use appropriate analytical methods that take non-participation into account, we need to understand its underlying mechanisms.
A recent large French retrospective cohort study was conducted by postal questionnaire among couples who had received in vitro fertilization (IVF) treatment. Information on all cohort members was collected in the IVF centers, allowing comparison of participants and non-participants. The main outcome of interest was parenthood project achievement after treatment in the inclusion IVF center (following further medical treatment elsewhere, spontaneous birth or adoption of a child). Achievement of the parenthood project may be associated with factors such as age, number of embryos obtained during IVF treatment or occurrence of a birth following medical treatment in the inclusion center. Comparison of these factors between participants and non-participants would enable us to discuss possible selection bias. We hypothesized that demographic, but not medical, characteristics may differ between contacted and non-contactable couples, whereas both may differ between respondents and non-respondents. Our objective was to assess factors associated with contact and response in an epidemiological study on the long-term outcome of couples after IVF treatment.
This study is based on the DAIFI study (Devenir Après Initiation d’un programme de FIV, outcome after IVF initiation), a retrospective cohort exhaustively including all couples who began an IVF program between 2000 and 2002 (n = 6,507) in one of the eight participating French IVF centers (the centers at Besançon University Hospital, Cochin Hospital, Caen, Marseille, Sèvres, Bois-Guillaume, Clermont-Ferrand and Montsouris). Initiation of an IVF program was defined as the first oocyte retrieval carried out at the center, regardless of whether the patient had had previous IVF treatment elsewhere.
The study received approval from the French Data Protection Authority in September 2005 (authorization number 05-1334).
Data collection was based on IVF center medical files and on a postal questionnaire filled in by patients.
Medical data on all eligible couples (n = 6,507) were obtained from the IVF centers. These data included sterility assessments for the couple (age, origin, type and duration of infertility), the number of IVF attempts in the center, information on these attempts (number of oocytes retrieved, number of embryos obtained, number of embryos transferred, pregnancy) and on the outcome of any ensuing pregnancies.
Data on the couples’ long-term outcome were collected via questionnaires filled in by the patients in 2008-2010. These data included sociodemographic information and the path followed by the couple before, during and after treatment in the inclusion center.
Contact rate was defined as the number of couples contacted among the total number of included couples, response rate as the number of respondents among the contacted couples, and participation rate as the number of respondents among the total number of included couples. As reasons for non-contact and for non-response were not necessarily the same, a two-step analysis strategy was conducted.
Firstly, contacted couples were compared with non-contactable couples. Secondly, among contacted couples, respondents to the postal survey were compared with non-respondents. Univariate and multivariate logistic regressions were conducted to assess associated factors: woman’s age (< 30, 30-34, 35-39, ≥ 40 years), year of the first attempt in the inclusion center (2000, 2001, 2002), inclusion center, origin of infertility (female or male factor, mixed, unexplained), the total number of embryos obtained at the first attempt (0-1, 2-5, > 5), number of attempts in the inclusion center (1, 2-4, > 4) and outcome of treatment in the center (live birth or not).
Statistical analyses were performed using STATA/SE 10.0 (Stata Press, College Station, TX, USA).
Among the 6,507 couples who began an IVF program in 2000-2002, the contact rate was 62% (n = 4,029). Among contacted couples, the response rate was 58% (n = 2,321). Therefore, 36% of the initial cohort participated in the postal study (Figure 1). The proportion of couples who had a child during IVF treatment was higher among contacted (44%) than among non-contactable couples (38%) and it was higher among respondents (53%) than among non-respondents (31%).
Factors associated with survey contact are presented in Table 1. The proportion of contacted couples increased from 56% in younger women (< 30 years) to 64% in older women (≥ 40 years) and varied according to inclusion center from 52% to 72%. The more recent the first oocyte retrieval, the higher the proportion of contacted couples, ranging from 58% to 66%. In univariate as well as in multivariate analyses, all factors were associated with the probability of contact, except origin of infertility. In multivariate analysis, a live birth after IVF remained associated with probability of contact with an OR of 1.34 (95% CI [1.20; 1.50]).
Factors associated with response to the postal survey are presented in Table 2. The proportion of respondents varied from 48% to 69% according to inclusion center. It increased with the total number of embryos transferred at first attempt and also appeared to be greater when the first oocyte retrieval was more recent. In multivariate analysis, a live birth following IVF was associated with the probability of response with an OR of 2.26 (95% CI [1.96; 2.60]).
To check the stability of our results among unsuccessfully treated couples, multivariate analyses were conducted a second time, but only among unsuccessfully treated couples, and thus after having removed the variable “result of IVF”. Multivariate analyses of factors associated with contact (n = 3,597) and of factors associated with response (n = 2,274) among unsuccessfully treated couples are presented in Tables 3 and 4, respectively. Results regarding the different variables (other than result of IVF) among unsuccessfully treated couples are very close to those observed for the whole cohort for probability of contact as well as for probability of response.
In the DAIFI cohort, a large retrospective cohort of 6,507 couples who began IVF treatment between 2000 and 2002, 36% of the initial cohort participated in a postal questionnaire survey 6 to 9 years later, after the first or second mailing; 38% of the cohort members could not be contacted and 26% were contacted but did not respond.
Thirty-eight percent of cohort members could not be contacted 6 to 9 years after beginning IVF treatment because they had moved to a new address. This proportion of lost to follow-up is similar to that observed in other studies of couples after IVF treatment. Among 1,614 eligible German couples, 44% could not be contacted 5 years after the birth of their ICSI child . Among 475 eligible English couples, 25.5% could not be contacted 4 to 10 years after referral to a fertility clinic . Obviously, the issue of loss to follow-up is not specific to IVF populations. For example, in the NEMESIS study investigating mental health in the general population in the Netherlands, 20% of attrition in the second wave was due to failure to locate or to contact respondents after only one year of follow-up . To mitigate the problem of contact, most prospective cohorts use processes such as annual update of address or contact details of relatives or friends [28, 29]. However, even with efforts to trace participants, in the Australian Longitudinal Study on Women’s Health 21% of 18- to 23-year-old women could not be contacted 4 years after the first survey . It is thus important to understand factors associated with non-contact. We observed a roughly linear relation between the woman’s age and the probability of contact, corresponding to greater mobility of younger couples, a finding which is in agreement with previous studies [5, 27]. As could logically be expected, women who had more recently begun IVF treatment in their inclusion center were more likely to be contacted (as they had shorter duration of follow-up), as were women with more numerous IVF attempts because they had left the center more recently and so also had a shorter duration of follow-up. The association between inclusion center and probability of contact may be linked to differences between centers in financial and human resources devoted to patient address update. It may also reflect the geographic location of the center as well as population dynamics, with mobility rates that can vary widely between regions. For instance, a change of address may be more likely in more urbanized areas . Lack of association between origin of infertility and contact suggests that medical factors do not have an impact on contact. Nevertheless, the association that we observed between the total number of embryos obtained at first attempt and probability of contact was an unexpected finding, as was the association with having a child after IVF treatment. The greater probability of contact among couples who had a live birth following treatment is particularly surprising, because a birth is one of the reasons for a change of address (need for one more bedroom). A higher rate of relocation among couples who did not have a child during IVF treatment could partly be due to a higher rate of couple separation. Such a hypothesis would need to be confirmed by further research.
Among the 4,029 couples contacted, 58% responded to the postal questionnaire. This rate appears similar to the few reported response rates among contacted couples in studies of IVF couples, and which ranged from 44% to 75% [23, 26, 30, 31]. In an IVF population, non-response could be linked to the physical and psychological burden of IVF treatment, especially when the treatment has not led to the expected live birth . However, some similar response rates have been reported in studies among young women. For example, in the Australian Longitudinal Study on Woman’s Health, 64% of women aged 18 to 23 years responded seven years after the first survey . Another recent study among uninsured women aged 15 to 44 years reported a response rate of 61% with a median follow-up of only 2.4 years . These results in fact led us to question the hypothesis that a lower response rate among IVF couples may be linked to the burden of treatment. Regarding factors associated with response, in our study an inverse-J relation was observed between the woman’s age and the probability of response. A similar relationship has been demonstrated between the woman’s age and the IVF live-birth rate [33, 34]. The inverse J-pattern between age and response suggests that age impacts as a medical factor on probability of response. Probability of response was also associated with inclusion center. Differences observed between centers may reflect in part couples’ feelings on their IVF treatment in the center, but probably also reflect sociodemographic characteristics of couples that may vary according to geographic location. Indeed, socioeconomic and educational levels are known to be associated with response rates in epidemiological studies [1, 8]. The trend toward a higher response rate among couples with unexplained infertility than in couples with infertility of female origin also suggests that demographic and medical factors influence contact and response in different ways. Our results may appear to differ from those of Cahill et al., who found that response rate to a postal questionnaire 4 to 10 years after referral to an IVF center was not significantly affected by the woman’s age, duration of infertility or ever having been pregnant or not . However, in this English study, lack of significant differences may be due to a lack of power, as the analyses were conducted on a small sample (n = 354). Participation was found to be strongly associated with birth of a child during treatment, indicating that there was a selection bias among the respondents to the postal survey. When the frequency of parenthood project achievement is being estimated, methods such as multiple imputation, that can adjusted for non-participation, should be used.
It is necessary to understand the mechanisms underlying contact and response in order to choose the appropriate methodology for analysis of the results of epidemiological surveys . To take into account attrition and potential bias, new methods are being developed but most rely on hypotheses that require an understanding of attrition mechanisms . Studies on attrition mechanisms are needed, especially as these mechanisms may vary according to the study population. In our study based on infertile couples treated by IVF, we found that an a priori hypothesis on attrition may be too simplistic and may underestimate potential bias. In our study, non-response as well as non-contact were linked to the outcome of interest. Attrition is a common issue in all health surveys and one that is rarely addressed in analysis. This study illustrates the importance of developing a study design that yields a minimum of information on the whole of the eligible population. In the context of growing use of analytical methods that take attrition into account (such as multiple imputation), we need to better understand the mechanisms that underlie attrition in order to choose the most appropriate method.
DAIFI Group members include: Institut National d’Etudes Démographiques (INED) - Institut National de la Santé et de la Recherche Médicale (INSERM) – Université Paris Sud XI: Elise de La Rochebrochard (national coordinator), Estelle Bailly, Jean Bouyer, Juliette Guibert, Henri Leridon, Patricia Thauvin, Laurent Toulemon, Pénélope Troude; Auvergne: Rusudan Peikrishvili, Jean-Luc Pouly (CHU Estaing, Clermont-Ferrand 63); Basse-Normandie: Isabelle Denis, Michel Herlicoviez (CHU Clémenceau, Caen 14); Franche-Comté: Christiane Joanne, Christophe Roux (CHR Saint-Jacques, Besançon 25); Haute-Normandie: Catherine Avril, Julie Roset (Clinique Saint-Antoine, Bois-Guillaume 76); Ile-de-France: Joëlle Belaisch-Allart, Olivier Kulski (Centre Hospitalier des 4 Villes, Sèvres 92); Jean-Philippe Wolf, Dominique de Ziegler (Cochin, Paris 75); Philippe Granet, Juliette Guibert (Institut Mutualiste Montsouris, Paris 75); Provence-Alpes-Côte d’Azur: Claude Giorgetti, Géraldine Porcu (Institut de Médecine de la Reproduction, IMR, Marseille 13).
Intracytoplasmic sperm injection
In vitro fertilization
Galea S, Tracy M: Participation rates in epidemiologic studies. Ann Epidemiol. 2007, 17: 643-653. 10.1016/j.annepidem.2007.03.013.
Altman DG: Statistics in medical journals: some recent trends. Stat Med. 2000, 19: 3275-3289. 10.1002/1097-0258(20001215)19:23<3275::AID-SIM626>3.0.CO;2-M.
Schneider KL, Clark MA, Rakowski W, Lapane KL: Evaluating the impact of non-response bias in the Behavioral Risk Factor Surveillance System (BRFSS). J Epidemiol Community Health. 2012, 66: 290-295. 10.1136/jech.2009.103861. Epub 2010 Oct 19
Brilleman SL, Pachana NA, Dobson AJ: The impact of attrition on the representativeness of cohort studies of older people. BMC Med Res Methodol. 2010, 10: 71-10.1186/1471-2288-10-71.
Littman AJ, Boyko EJ, Jacobson IG, Horton J, Gackstetter GD, Smith B, Hooper T, Wells TS, Amoroso PJ, Smith TC: Assessing nonresponse bias at follow-up in a large prospective cohort of relatively young and mobile military service members. BMC Med Res Methodol. 2010, 10: 99-10.1186/1471-2288-10-99.
Kristman V, Manno M, Cote P: Loss to follow-up in cohort studies: how much is too much?. Eur J Epidemiol. 2004, 19: 751-760.
Fewtrell MS, Kennedy K, Singhal A, Martin RM, Ness A, Hadders-Algra M, Koletzko B, Lucas A: How much loss to follow-up is acceptable in long-term randomised trials and prospective studies?. Arch Dis Child. 2008, 93: 458-461. 10.1136/adc.2007.127316.
Young AF, Powers JR, Bell SL: Attrition in longitudinal studies: who do you lose?. Aust N Z J Public Health. 2006, 30: 353-361. 10.1111/j.1467-842X.2006.tb00849.x.
Morton LM, Cahill J, Hartge P: Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol. 2006, 163: 197-203.
Razafindratsima N, Kishimba N, COCON Group: Attrition in the COCON cohort between 2000 and 2002. Popul (English). 2004, 59: 357-385.
Plewis I, Ketende S: Millennium Cohort Study: Technical Report on Response. 2006, London: Centre for Longitudinal Studies, Institute of Education
Ketende S: Millennium Cohort Study: Technical Report on Response. 2008, London: Centre for Longitudinal Studies, Institute of Education, 2
Nohr EA, Frydenberg M, Henriksen TB, Olsen J: Does low participation in cohort studies induce bias?. Epidemiology. 2006, 17: 413-418. 10.1097/01.ede.0000220549.14177.60.
Jacobsen TN, Nohr EA, Frydenberg M: Selection by socioeconomic factors into the Danish National Birth Cohort. Eur J Epidemiol. 2010, 25: 349-355. 10.1007/s10654-010-9448-2.
Banks E, Redman S, Jorm L, Armstrong B, Bauman A, Beard J, Beral V, Byles J, Corbett S, Cumming R, et al: Cohort profile: the 45 and up study. Int J Epidemiol. 2008, 37: 941-947.
Mealing NM, Banks E, Jorm LR, Steel DG, Clements MS, Rogers KD: Investigation of relative risk estimates from studies of the same population with contrasting response rates and designs. BMC Med Res Methodol. 2010, 10: 26-10.1186/1471-2288-10-26.
Lee C, Dobson AJ, Brown WJ, Bryson L, Byles J, Warner-Smith P, Young AF: Cohort Profile: the Australian Longitudinal Study on Women’s Health. Int J Epidemiol. 2005, 34: 987-991. 10.1093/ije/dyi098.
Goldberg M, Chastang JF, Leclerc A, Zins M, Bonenfant S, Bugel I, Kaniewski N, Schmaus A, Niedhammer I, Piciotti M, et al: Socioeconomic, demographic, occupational, and health factors associated with participation in a long-term epidemiologic survey: a prospective study of the French GAZEL cohort and its target population. Am J Epidemiol. 2001, 154: 373-384. 10.1093/aje/154.4.373.
Goldberg M, Chastang JF, Zins M, Niedhammer I, Leclerc A: Health problems were the strongest predictors of attrition during follow-up of the GAZEL cohort. J Clin Epidemiol. 2006, 59: 1213-1221. 10.1016/j.jclinepi.2006.02.020.
Olivius C, Friden B, Borg G, Bergh C: Why do couples discontinue in vitro fertilization treatment? a cohort study. Fertil Steril. 2004, 81: 258-261. 10.1016/j.fertnstert.2003.06.029.
Schmidt L: Psychosocial burden of infertility and assisted reproduction. Lancet. 2006, 367 (9508): 379-380. 10.1016/S0140-6736(06)68117-8.
Filetto JN, Makuch MY: Long-term follow-up of women and men after unsuccessful IVF. Reprod Biomed Online. 2005, 11: 458-463. 10.1016/S1472-6483(10)61141-8.
Hammarberg K, Astbury J, Baker H: Women’s experience of IVF: a follow-up study. Hum Reprod. 2001, 16: 374-383. 10.1093/humrep/16.2.374.
Tate AR, Jones M, Hull L, Fear NT, Rona R, Wessely S, Hotopf M: How many mailouts? Could attempts to increase the response rate in the Iraq war cohort study be counterproductive?. BMC Med Res Methodol. 2007, 7: 51-10.1186/1471-2288-7-51.
Ludwig AK, Katalinic A, Jendrysik J, Thyen U, Sutcliffe AG, Diedrich K, Ludwig M: Spontaneous pregnancy after successful ICSI treatment: evaluation of risk factors in 899 families in Germany. Reprod Biomed Online. 2008, 17: 403-409. 10.1016/S1472-6483(10)60225-8.
Cahill DJ, Meadowcroft J, Akande VA, Corrigan E: Likelihood of natural conception following treatment by IVF. J Assist Reprod Genet. 2005, 22: 401-405. 10.1007/s10815-005-6655-y.
de Graaf R, Bijl RV, Smit F, Ravelli A, Vollebergh WA: Psychiatric and sociodemographic predictors of attrition in a longitudinal study: The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Am J Epidemiol. 2000, 152: 1039-1047. 10.1093/aje/152.11.1039.
Lee C, Dobson A, Brown W, Adamson L, Goldsworthy J: Tracking participants: lessons from the Women’s Health Australia Project. Aust N Z J Public Health. 2000, 24: 334-336. 10.1111/j.1467-842X.2000.tb01580.x.
Ware RS, Williams GM, Aird RL: Participants who left a multiple-wave cohort study had similar baseline characteristics to participants who returned. Ann Epidemiol. 2006, 16: 820-823. 10.1016/j.annepidem.2006.01.008.
Pinborg A, Hougaard CO, Nyboe Andersen A, Molbo D, Schmidt L: Prospective longitudinal cohort study on cumulative 5-year delivery and adoption rates among 1338 couples initiating infertility treatment. Hum Reprod. 2009, 24: 991-999.
Rajkhowa M, McConnell A, Thomas GE: Reasons for discontinuation of IVF treatment: a questionnaire study. Hum Reprod. 2006, 21: 358-363.
Fanarjian N, Drostin C, Garrett J, Montalvo A: Does the provision of free intrauterine contraception reduce pregnancy rates among uninsured low-income women? A cohort study: a two North Carolina clinics. Contraception. 2012, 85: 160-165. 10.1016/j.contraception.2011.06.002. Epub 2011 Jul 14
Templeton A, Morris JK, Parslow W: Factors that affect outcome of in-vitro fertilisation treatment. Lancet. 1996, 348 (9039): 1402-1406. 10.1016/S0140-6736(96)05291-9.
Soullier N, Bouyer J, Pouly JL, Guibert J, de La Rochebrochard E: Effect of the woman’s age on discontinuation of IVF treatment. Reprod Biomed Online. 2011, 22: 496-500. 10.1016/j.rbmo.2011.01.013.
MacDonald SE, Newburn-Cook CV, Schopflocher D, Richter S: Addressing nonresponse bias in postal surveys. Public Health Nurs. 2009, 26: 95-105. 10.1111/j.1525-1446.2008.00758.x.
Graham JW: Missing data analysis: making it work in the real world. Annu Rev Psychol. 2009, 60: 549-576. 10.1146/annurev.psych.58.110405.085530.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/12/104/prepub
None of the authors have any conflict of interest.
This study was not sponsored or funded by industry. It received funding only from French government agencies.
P. Troude designed the study’s analytic strategy, conducted literature review, conducted the statistical analysis, and drafted the manuscript. E. Bailly participated in data collection, reviewed the study’s analytic strategy and reviewed the manuscript. J. Guibert coordinated the study in fertility centers, reviewed the study’s analytic strategy and results, and reviewed the manuscript. J. Bouyer designed the study’s analytic strategy, reviewed results, and reviewed the manuscript. E. de La Rochebrochard designed the study, designed the study’s analytic strategy, and drafted the manuscript. She had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis All members of the DAIFI group reviewed the manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Troude, P., Bailly, E., Guibert, J. et al. Who does not participate in a follow-up postal study? a survey of infertile couples treated by in vitro fertilization. BMC Med Res Methodol 12, 104 (2012). https://doi.org/10.1186/1471-2288-12-104
- Postal survey
- Response rate