Effects of different invitation strategies on participation in a cohort study of Iranian public sector employees: a cluster randomized trial
BMC Medical Research Methodology volume 21, Article number: 206 (2021)
Identifying strategies to optimize participation in health studies is one of the major concerns for researchers. The purpose of this study was to evaluate the efficiency of different invitation strategies on participation rate in the Employees’ Health Cohort Study of Iran (EHCSIR).
Two cluster-randomized trials were carried out to assess the outcomes of different invitation strategies. In the first phase, 7 units with 1880 employees (3 hospitals, 3 health centers, and 1 office) were assigned to the three parallel modes of invitation: 1) invitation letter, 2) phone call and 3) Short Message Service (SMS). In the second phase, 6 hospitals with 1633 employees were allocated to two invitation methods: 1) invitation letter, 2) invitation letter plus EHCSIR project introduction video. All groups were followed up by phone calls. A logistic mixed-effects model was used to compare the effectiveness of the strategies. The cost-effectiveness of the interventions was also compared.
In the first phase, the participation rates in the invitation letter, phone call, and SMS groups were 27.04% (182/673), 21.55% (131/608), and 22.54% (135/599), respectively. Using an invitation letter was significantly more successful than SMS (Adjusted Odds Ratio = 1.80, 95% CI =1.14 to 2.85). Average Cost-Effectiveness Ratios (ACER) were $1.37, $1.42, and $1.55 for the invitation letter, phone call, and SMS, respectively. In the second phase, adding a project introduction video to the invitation letter did not significantly influence the participation rate (Adjusted OR = 0.58, 95% CI =0.24 to 1.36). The ACER was $1.21 for the invitation letter only and $2.01 for the invitation letter plus the introduction video.
In comparison with the phone call and SMS, the invitation letter is the most effective invitation method for public sector employees to participate in a cohort study. Sending an introduction video did not significantly increase the participation rate compared to sending an invitation letter only.
The ability to attract participants is crucial to conduct high-quality research. Failure to achieve the goal of recruiting potential participants can threaten the validity of a study [1, 2]. This failure may lead to methodological, practical, and ethical challenges [3, 4]. Nowadays, recruiting participants has become more difficult, and the response rate in epidemiological studies has decreased dramatically over the past decades [5, 6]. Even in studies that succeed in recruiting a large number of potential participants, the participation level remains low [4, 7]. The participation rates in studies that have a biobank and blood sample collection are less than in other studies .
Literature suggests several strategies that can enhance participation rate. For instance, in several studies, paying incentives was shown effective in increasing participation [2, 9, 10], but it imposes more additional financial burden than direct invitation methods . Also, it is recommended that investigators use repeated attempts using different strategies in the invitation to health studies [10, 12]. Because of the variability of the participation rate in epidemiological studies based on the target population and the type of study, a comprehensive investigation of recruitment and maintenance should be one part of the design of any research .
The objectives of the current study were to explore the impact of different invitation strategies on participation in the EHCSIR project and the cost-effectiveness of those methods.
The Employees’ Health Cohort Study of Iran (EHCSIR) is a cohort study of Iranian public sector employees mainly from schools, hospitals and health centers affiliated with the Iran University of Medical Science and the Ministry of Health and Medical Education. The purpose of the study is to identify occupational and non-occupational risk factors for non-communicable diseases, including cardiovascular diseases, cancers, respiratory diseases, vision and hearing disorders, and common psychiatric disorders. All participants take part in several interviews: clinical (past medical history, medications history, smoking), general (migration, ethnicity, lifestyle, and physical activity), dietary habits (including a Food Frequency Questionnaire), job and life stress (job demand/ content, effort-reward imbalance, satisfaction in domains of life), workplace and social health (work performance, quality of life, social capital, workplace social capital), psychological characteristics (personality traits, sleep disorders, smartphone addiction, internet problematic use, aggression, depressive and anxiety disorders). Blood and urine samples are taken for both ad-hoc examinations and storage in a biobank. Blood pressure measurement, electrocardiogram, comprehensive optometric and audiometry assessments, spirometry, anthropometry, and Body Composition Analysis are performed. All of the interviews, clinical and laboratory assessments are performed during 6–7 h on the visit day, and all the deliverable results of investigations are presented to the participants on the same day.
The study population of EHCSIR is more than 15,000 employees working in 43 different work units. Each unit has a liaison officer who is working at the same unit and is nominated by the director of the unit to coordinate arrangements for the participation of the unit’s employees in EHCSIR. Liaison officers are trained in a two-day workshop to learn about the study and the recruitment process. The liaison officers invite colleagues from their own work unit to participate in the study and introduce them to the recruitment unit of EHCSIR project. The recruitment started in July 2017, and the most important challenge of the described recruitment procedure was the low and unmeasurable response rate.
We conducted two cluster randomized trials, the first with three and the second with two parallel intervention groups.
In the first phase, 7 clusters with 1880 employees (3 hospitals, 3 health centers, 1 office) were selected. One hospital and one health center were randomly assigned to each invitation group and one administrative unit to the SMS group that had smaller sample size. In the second phase, 6 hospitals with 1633 employees were randomly assigned to two invitation groups.
As shown in Fig. 1, the invitation procedure consists of several steps. A standard operational protocol was developed for each invitation strategy. A member of the present study made all the contacts with potential participants. In all groups, the content of the provided information about EHCSIR was the same (Additional File 1). Also, liaison officers were trained to follow every unit’s invitation protocol. We have sent the printed invitation letter to the liaison officer and he/she has given them to employees. The demographic and job profile data, besides mobile or home phone numbers were gained from each unit.
The only difference between intervention groups was the mode of the initial invitation.
The first phase interventions consisted of.
1) Invitation letter: the personalized invitation letter that was signed by the EHCSIR’s principal investigator. The letter was printed on EHCSIR letterhead and envelope.
2) Phone call: the employees were invited by phone call.
3) SMS: two identical invitation text messages were sent 48 h apart through the organizational SMS system.
The second phase invitation modes include.
1) Invitation letter: The letter was the most effective strategy based on the first phase results.
2) Invitation letter plus EHCSIR project introduction video: in addition to the letter, the 7-min EHCSIR introduction video was sent to the employees via WhatsApp.
Follow-up phone call: In all invitation strategies, those who did not respond actively to the invitation within 1 week were followed up by a phone call. During this contact, based on the group, the recruiter asked whether they had read the invitation SMS, letter, or watched the video. The recruiter had to make at least three calls.
The sample size was calculated as 600 employees in each group to determine a 20% difference (13%  to 33% ) in the response rate between the two groups. Intra-cluster correlation coefficient, average cluster size, confidence level of and power were considered 0.02, 200 , 95, and 80%, respectively.
Two phases had independent samples and were analyzed independently. Baseline characteristics of employees were summarized by the invitation group. Mean, standard deviation (SD) and median, interquartile range (IQR) were reported for continuous variables, whereas frequencies and proportions were reported for categorical variables. The baseline comparability between the invitation groups was assessed using χ2 for categorical variables, in both phases and the Kruskal-Wallis and Mann-Whitney for the continuous variables since their distributions were non-normal in the first phase and the second phase, respectively.
The primary outcome was the participation in EHCSIR as a binary outcome (participation and non-participation) up to a maximum of 3 months after the invitation that was the first contact with the person. The response rate for each intervention group with exact binomial confidence intervals of 95% was calculated. The logistic mixed-effects model with random effects on type of the unit (hospital, health center, and office) and the unit levels to compare the effectiveness of different invitation strategies were applied to estimate the odds ratio of the participation and its 95% confidence interval. The intention to treat analysis approach was used primarily. In a secondary analysis, per-protocol, per-contacted person, analysis approaches were applied.
Intention to treat analysis included all people who were allocated to each invitation group.
The per-contacted analysis included employees that we contacted either in the initial invitation or in the follow-up phone call because we have not called with all of them. The major reason was employees did not answer the phone call. Another reason was the telephone numbers of employees either have been not registered in the database or have been wrong, despite the efforts of the liaisons.
The per-protocol analysis in the invitation letter and SMS groups included contacted employees that had read the letter or SMS before the follow-up call, in the invitation letter plus video group included people who had read the letter and watched the video before the follow-up call, and in the phone group, included all contacted people.
Per-treat analysis, in the first phase, in the invitation letter and SMS groups included contacted employees who had read the letter or SMS before the follow-up call. But people who had not read the letter or SMS were considered phone group. In the phone group, included all contacted people.
Per-treat analysis, in the second phase, in the invitation letter group included contacted employees who had read the letter before the follow-up call. In the invitation letter plus video group, it included people who had read the invitation letter and had watched the video, people who had read the invitation letter but had not watched the video were considered the invitation letter group.
We considered the gender, age, job category, and employment period as individual-level predictors, also the workplace distance from the study center and the Workplace Social Capital (WPSC) as group-level predictors. For the WPSC score, we used the WPSC questionnaire average score of the participants of each center that was completed in EHCSIR. Literature has shown age, gender, job, and distance from the study center were predictors of participation in studies [10, 15,16,17]. However, we checked the distribution of baseline characteristics between groups. Therefore, we reported our unadjusted and adjusted results for these variables and WPSC. Because of collinearity between the age and the employment duration variables, we just kept the age in the model.
Pairwise comparisons between the invitation letter and phone call have been made using post-estimation commands. All statistical analyses were performed using Stata version 14 and 0.05 was used as the threshold for statistical significance.
The total cost of each invitation strategy was determined by summing the cost of all relevant material, including letter paper, envelope, printing, sending the letter, sending SMS, Internet, invitation phone calls, all follow-up calls, and the staff cost. It should be noted that we did not consider the costs of producing the video since the video already had been produced; we used the video before releasing it. Average Cost-Effectiveness Ratio (ACER) was defined as the sum of individual costs divided by the number of participants in EHCSIR.
The flow diagram of two phases was summarized in Fig. 2. Out of 1880 employees invited to the first phase, 1205 (64.10%) were female. The mean and median age of the employees were 42.23 (SD =8.03) and 41(IQR = 36–48) years.
In the second phase, out of 1633 invited employees, 996 (59.15%) were women. Their mean and median age were 42.21 (SD = 7.96) and 41(IQR = 36–48).
Table 1 shows a baseline comparison of the intervention groups. Distributions of baseline characteristics between intervention groups were different significantly (p < 0.05). It should be noted that these differences were not important in a medical sense.
Number of calls per respondent
A few numbers attended actively to the invitation within 1 week. In the first phase, we have not accessed the telephone number of 30(1.60%) employees: 1(0.17%), 4(0.66%), and 25(3.71%) in the SMS, phone, and invitation letter group, respectively. The maximum number of follow-up calls was 5 times. Of those who participated, 64.44, 87.63, and 75.27% attended after the first invitation call who were invited by the SMS, the phone call, and the invitation letter, respectively (further details available in Additional file 2: Appendix 1, Table 1).
In the second phase, we have not accessed the telephone number of 98(6.00%) employees: 9(1.06%) and 89(11.37%) in the invitation letter plus video and the invitation letter group, respectively. The maximum number of follow-up calls was 5 times. Of those who participated, 92.42 and 89.08% participated after the first invitation call who were invited by the invitation letter alone and the invitation letter plus video, respectively (further details available in Additional file 2: Appendix 1, Table 2).
Intention to treat analysis
The overall participation rate in the current study was 25.22%. The effectiveness of the interventions on the participation rates are given in Table 2. The invitation letter was the highest response rate (27.04%) compared to the phone call and SMS strategies. Also, there was a small difference between the participation rate on the phone call and SMS methods (21.55% vs. 22.54).
The participation rate by the invitation letter alone and the invitation letter plus the EHCSIR project introduction video were 33.72 and 20.47%, respectively.
The results of the logistic mixed-effects models showed that in the first phase, the invitation letter compared to SMS significantly increased participation (Adjusted OR = 1.80, 95% CI = 1.14–2.85). The comparison between the invitation letter versus phone call showed no significant difference between their effect on participation (Unadjusted OR = 1.44, 95%CI = 0.52–3.97), (Adjusted OR = 2.33, 95%CI = 0.64–8.49) (further details available in Additional file 2: Appendix 2).
In the second phase, the participation rate of those who were sent the EHCSIR project introduction video together with the invitation letter was not significantly different from the invitation letter-only group (Adjusted OR = 0.58, 95% CI = 0.24–1.36).
The analysis based on the contacted people showed that the invitation letter was significantly better than the SMS (Adjusted OR = 1.67, 95% CI = 1.05–2.66) (see Table 2). The comparison between the invitation letter and phone call showed no significant difference between the effect of invitation letter and phone call strategies on participation (Unadjusted OR = 1.48, 95%CI = 0.59–3.72), (Adjusted OR = 2.12, 95%CI = 0.66–6.82).
In the second phase, adding video to the invitation letter could not significantly increase the response versus only the invitation letter (Adjusted OR = 0.41, 95% CI = 0.15–1.07) (see Table 2).
In the first phase, among contacted people in the SMS and the invitation letter groups, 409 and 417 people mentioned that they had read the SMS and the invitation letter before the follow-up contact, respectively. We considered people who read the SMS or the invitation letter, also all contacted people (490) in the phone group, as the people who had followed the invitation protocol.
The analysis based on the adherence to invitation protocol showed that the phone call (Adjusted OR = 0.65, 95% CI = 0.22–1.92) and invitation letter (Adjusted OR = 1.60, 95% CI = 0.98–2.64) did not significantly affect the participation compared with the SMS method (see Table 2). The comparison between the invitation letter and phone call showed that there was no significant difference between them (Unadjusted OR = 1.74, 95%CI = 0.71–4.26), (Adjusted OR = 2.48, 95%CI = 0.82–7.47). The Workplace distance from EHCSIR center and WPSC did not affect the participation rate significantly (P > 0.05).
In the second phase, among contacted people in the invitation letter plus video group, 259 people had read the letter and watched the video before the follow-up contact. Also, in the invitation letter group, 510 people had read the invitation letter before the follow-up contact. We considered them as the person who has followed the invitation protocol.
The analysis based on the adherence to invitation protocol showed that adding the study introduction video to the invitation letter significantly reduced the response versus the invitation letter only (Adjusted OR = 0.27, 95% CI = 0.08–0.85) (see Table 2).
In the first phase, among contacted people in the letter (526) and SMS (499) groups, people who had not read the letter or SMS were considered the phone group. The per-treat analysis showed that participation in the invitation letter group was significantly higher than the SMS group (Adjusted OR = 1.59, 95% CI = 1.00–2.53), but participation in the phone call group was significantly lower than SMS group (Adjusted OR = 0.45, 95% CI = 0.27–0.73) (see Table 2). The comparison between the invitation letter and phone call showed that using the invitation letter increased participation in the study significantly compared with the phone call strategy (Unadjusted OR = 3.47, 95%CI = 2.19–5.48), (Adjusted OR = 3.55, 95%CI = 2.24–5.64).
In the second phase, among 850 whom people were invited by the invitation letter plus video, only 259 (30.47%) read the letter and watched the video, 329 (47.89%) just read the letter; as a result, in per-treat analysis, these people were considered the invitation letter plus video and the invitation letter groups, respectively. In the invitation letter group, 510 (71.97%) read the invitation letter. Finally, 839 and 259 people were considered as the invitation letter and the invitation letter plus video group, respectively.
The per-treat analysis showed that adding video to the invitation letter did not significantly influence the response versus only the invitation letter (Adjusted OR = 1.27, 95% CI = 0.87–1.85) (see Table 2).
Table 3 shows cost-effectiveness analysis. In the first phase, the average cost per participation was $1.37, $1.42, and $1.55 for the invitation letter, phone call, and SMS groups, respectively.
In the second phase, the average cost per participation was $1.21 and $2.01 for the invitation letter, and the invitation letter plus video groups, respectively.
This study assessed the impact of several invitation strategies to participate in an employees’ cohort study. We achieved the highest response rate using the invitation letter. The invitation letter successfully improved participation compared with SMS. However, the phone call did not significantly influence participation compared with the SMS method. The invitation letter had the lowest average cost per participation in the study. Sending the study introduction video did not significantly affect participation compared with sending only the invitation letter. Those who watched the video were more likely to participate than those who did not. Perhaps it is because those who are intrinsically interested in participating in the study are more likely to watch a video about the study.
Based on the social exchange theories, prior notification mailings help build confidence in a study . Besides, behavior theory suggests that the personalized letter could strengthen the study’s relationship with participants and, as a result, increases the participation rate . In several large studies, mass mailing was a critical invitation strategy [7, 13, 18].
In our study, the participation rates in phases I and II between the two invitation letter groups were different. It could be because our target population was employees of one university who could share their experiences and information during progressing of the EHCSIR project.
We could not compare our results with previous studies accurately because evidence about the effectiveness and cost-effectiveness of invitation strategies in cohort studies is limited [2, 11]. Moreover, the participation rate and the reasons that people take part in health studies are variable according to the characteristics of the target population [3, 19]. However, consistent with our study, some studies have suggested that invitation letter was more cost-effective . In another study, invitation by two letters was more effective and more cost-effective than invitations by one letter. Also, they reported the highest efficiency for one letter plus phone call in people whose phone number was available .
In our study, it was not possible to assess the phone reminder effect because we used phone call follow-up as one of our invitation steps in all our intervention groups. The previous studies [1, 6, 9, 15] indicated that reminder and follow-up, especially phone call reminders, were effective in increasing the response rate. Though, in another study, sending a reminder letter did not have a significant impact on the response rate .
Another study showed that there is no difference between sending a letter and an e-mail, but the cost of sending a letter was higher . The cost per participation in all invitation strategies in our study was lower in all invitation strategies in our study than in other research [6, 10, 21].
Age, job category, and distance from the study center were the determinants of the participation rate. By controlling other factors, participation has also increased with age. In most studies, despite the differences in the target population, the relationship between the age and participation rate has been shown, and it has been concluded that the likelihood of participation of older people has been more than younger people [15, 16]. In this study, the women’s participation rate was slightly higher than men, but no significant association was observed. This result is consistent with a study by Banks et al. , but in other studies, the participation of women has been significantly higher than men [10, 16].
The strengths of this study are its methodology, sufficient sample size, and availability of demographic information of employees without any missing data. Besides, the cost-effectiveness of the interventions has been calculated.
The main limitation of this study was the low number of clusters; in this regard, the balance between intervention groups did not meet. Nevertheless, we had controlled known and measured predictors of participation; other kinds of predictors have not been controlled obviously. As a result, our results suffer from somewhat uncertainty.
Compared to the phone call and SMS, the invitation letter is the most successful method of inviting public sector employees to participate in a cohort study. Also, the invitation letter was the most cost-effective strategy in comparison with the phone call and SMS. There is a no significant impact on sending the study introduction video in increasing the likelihood of participation. To make sure about the video’s effect on the response rate, it seems that we should be looking for a strategy that increases the likelihood of watching the video.
Availability of data and materials
The datasets generated and analyzed during the current study are not publicly available because of containing information that could potentially identify study participants. However, de-identified data are available from the corresponding author on reasonable request.
Employees’ Health Cohort Study of Iran
Short Message Service
Average Cost-Effectiveness Ratio
Incremental Cost-Effectiveness Ratio
Workplace Social Capital
Intention to Treat
- Med (IQR):
Median (Inter Quartile Range)
Galea S, Tracy M. Participation rates in epidemiologic studies. Ann Epidemiol. 2007;17(9):643–53.
Mapstone J, Elbourne D, Roberts I. Strategies to improve recruitment to research studies. Cochrane Database Syst Rev. 2007;2:Mr000013.
Hays J, Hunt JR, Hubbell FA, Anderson GL, Limacher M, Allen C, et al. The women's health initiative recruitment methods and results. Ann Epidemiol. 2003;13(9):S18–77.
Hinshaw LB, Jackson SA, Chen MY. Direct mailing was a successful recruitment strategy for a lung-cancer screening trial. J Clin Epidemiol. 2007;60(8):853–7.
Wong AD, Kirby J, Guyatt GH, Moayyedi P, Vora P, You JJ. Randomized controlled trial comparing telephone and mail follow-up for recruitment of participants into a clinical trial of colorectal cancer screening. Trials. 2013;14(1):1–5.
Treweek S, Barnett K, MacLennan G, Bonetti D, Eccles MP, Francis JJ, et al. E-mail invitations to general practitioners were as effective as postal invitations and were more efficient. J Clin Epidemiol. 2012;65(7):793–7.
Tworoger SS, Yasui Y, Ulrich CM, Nakamura H, LaCroix K, Johnston R, et al. Mailing strategies and recruitment into an intervention trial of the exercise effect on breast cancer biomarkers. Cancer Epidemiol Biomark Prev. 2002;11(1):73–7.
García-Closas M, Egan KM, Abruzzo J, Newcomb PA, Titus-Ernstoff L, Franklin T, et al. Collection of genomic DNA from adults in epidemiological studies by buccal cytobrush and mouthwash. Cancer Epidemiol Biomark Prev. 2001;10(6):687–96.
Page A, Morrell S, Chiu C, Taylor R, Tewson R. Recruitment to mammography screening: a randomised trial and meta-analysis of invitation letters and telephone calls. Aust N Z J Public Health. 2006;30(2):111–8.
Boyd A, Tilling K, Cornish R, Davies A, Humphries K, Macleod J. Professionally designed information materials and telephone reminders improved consent response rates: evidence from an RCT nested within a cohort study. J Clin Epidemiol. 2015;68(8):877–87.
Huynh L, Johns B, Liu S-H, Vedula SS, Li T, Puhan MAJCT. Cost-effectiveness of health research study participant recruitment strategies: a systematic review. Clinical Trials. 2014;11(5):576–83.
Chen K-F, Colantuoni E, Siddiqi F, Dinglas VD, Sepulveda KA, Fan E, et al. Repeated attempts using different strategies are important for timely contact with study participants. J Clin Epidemiol. 2011;64(10):1144–51.
Banks E, Herbert N, Rogers K, Mather T, Jorm L. Randomised trial investigating the relationship of response rate for blood sample donation to site of biospecimen collection, fasting status and reminder letter: the 45 and up study. BMC Med Res Methodol. 2012;12(1):1–11.
Richards SH, Bankhead C, Peters TJ, Austoker J, Hobbs FD, Brown J, et al. Cluster randomised controlled trial comparing the effectiveness and cost-effectiveness of two primary care interventions aimed at improving attendance for breast screening. J Med Screen. 2001;8(2):91–8.
Keeble C, Baxter P, Barber S, Law G. Participation rates in epidemiology studies and surveys: a review 2005-2007. Internet J Epidemiology. 2016;14(1):1–14.
Hara M, Higaki Y, Imaizumi T, Taguchi N, Nakamura K, Nanri H, et al. Factors influencing participation rate in a baseline survey of a genetic cohort in Japan. J Epidemiology. 2010;20(1):40–5.
Banks E, Herbert N, Mather T, Rogers K, Jorm L. Characteristics of Australian cohort study participants who do and do not take up an additional invitation to join a long-term biobank: the 45 and up study. BMC Res Notes. 2012;5(1):1–6.
Toledano MB, Smith RB, Brook JP, Douglass M, Elliott P. How to establish and follow up a large prospective cohort study in the 21st century-lessons from UK COSMOS. PLoS One. 2015;10(7):e0131521.
Nobile H, Vermeulen E, Thys K, Bergmann MM, Borry P. Why do participants enroll in population biobank studies? A systematic literature review. Expert Rev Mol Diagn. 2013;13(1):35–47.
Bankhead C, Richards S, Peters T, Sharp D, Hobbs F, Brown J, et al. Improving attendance for breast screening among recent non-attenders: a randomised controlled trial of two interventions in primary care. J Med Screen. 2001;8(2):99–105.
Sahlqvist S, Song Y, Bull F, Adams E, Preston J, Ogilvie D. Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial. BMC Med Res Methodol. 2011;11(1):1–8.
The authors would like to thank sincerely all those who participated in the study. Special thanks to the Liaison officers of work units, who helped us in the execution of this study. Thanks to the EHCSIR’s personnel, who helped us in the execution of this study.
The Vice-Chancellor for Research & Technology of Iran University of Medical Sciences funded this study.
Ethics approval and consent to participate
This study was approved by the ethics committee of Iran University of Medical Science (Ethical codes: #IR.IUMS.REC.1398.281). All participants provided written informed consent prior to enrolling in the study. The data of Participants kept confidential and was only accessible to the study investigators. All methods were carried out in accordance with relevant guidelines and regulations.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rajabzadeh, R., Janani, L. & Motevalian, S.A. Effects of different invitation strategies on participation in a cohort study of Iranian public sector employees: a cluster randomized trial. BMC Med Res Methodol 21, 206 (2021). https://doi.org/10.1186/s12874-021-01405-8