Engaging military couples in marital research: does requesting referrals from service members to recruit their spouses introduce sample bias?

Abstract

Background

While enrolling dyads in research studies is not uncommon, there is limited literature on the utility of different recruitment strategies and the resulting selection biases. This paper examined two recruitment strategies used to enroll military couples in a longitudinal study, assessing the impact of both strategies on the representativeness of the final study sample.

Method

Descriptive and bivariate analyses were conducted to 1) identify characteristics associated with spouse referral, 2) compare response rates based on recruitment strategy and assess whether recruitment strategy modified correlates of response propensity among spouses, and 3) assess whether referred spouse characteristics differed from non-referred spouses in the final sample. The study sample consisted of married US service members with 2–5 years of military service as of October 2011 and their spouses.

Results

Service members who referred their spouses to participate in the Millennium Cohort Family Study were more likely to be male, have children, serve in the Army, and have combat deployment experience than those who did not refer their spouse. Nearly two-thirds (n = 5331, 64.9%) of referred spouses participated in the Family Study, compared with less than one-third (n = 3458, 29.5%) of directly contacted spouses. Spouse characteristics also differed significantly between recruitment groups.

Conclusions

Overall results suggest that minimal bias was introduced by using a referral recruitment methodology. Service members appeared to be more likely to refer their spouses if they perceived the research topic as relevant to their spouse, such that male service members with combat deployment experience were more likely to refer female spouses caring for multiple children. Referred spouses were significantly more likely to respond to the Millennium Cohort Family Study survey than those who were directly contacted; however, the overall success rate of using a referral strategy was less than recruiting spouses through direct contact. Differences between referred spouses and spouses contacted directly mirrored service member referring characteristics.

Background

Recognizing the importance of human relationships and social interactions, researchers may seek to recruit and study related individuals, such as family members, co-habitants, co-workers, and sexual partners, rather than unrelated individuals. Indeed, many research questions are optimally addressed, or can only be addressed, by using information from dyads. For instance, data from both spouses of a couple are necessary to investigate interpersonal aspects of marriage [1, 2]; other types of family dynamics are most appropriately explored using parent-child or sibling dyads [3, 4]. Medical researchers interested in patient outcomes may study patient-caregiver dyads [5, 6] and often conduct studies of twins in order to tease apart genetic and environmental influences on health [7]. Despite their importance, dyadic research designs present a multitude of methodological complexities [5], beginning with the challenge of recruiting a representative sample of participant pairs.

The recruitment of dyads generally involves many of the same strategies used in traditional subject recruitment, such as placing advertisements in newspapers, approaching patients in hospitals or clinics, posting flyers, or taking advantage of samples of convenience (e.g., students), and some of these may limit the generalizability of findings [1, 8,9,10,11]. Furthermore, in dyadic studies researchers must somehow arrange to solicit both members of each potential dyad and ensure in the final dataset that both partners have volunteered together. Unfortunately, many research reports fail to provide valuable details concerning these strategies, and they rarely report rates of nonresponse—for instance, for husbands and wives separately [8, 10], information useful in determining the generalizability of the sample. It is even less common to have some form of public record available for the entire recruitment population (e.g., marriage license records), so the representativeness of the sample can be empirically determined and reported [8].

It is not uncommon to begin the recruitment of a dyad by soliciting the participation of one person and then subsequently enlisting their help in approaching their partner through a referral. However, there is scant literature on the utility of referrals in subject recruitment and the potential selection biases that may result from it. In fact, we have only found a few research reports that detail the process of engaging dyads in any manner, typically where recruitment was particularly difficult or where engaging and retaining the dyad in an educational or treatment program was important [9, 10, 12,13,14]. Unfortunately, even these studies generally did not address the potential for sample bias introduced through differential recruiting methods, such as spouse referral.

One notable exception was a study of recruitment strategy and decision-making in fertility treatment conducted by Preloran et al. [10]. In this study, a woman’s involvement in recruiting her male partner was related to marital quality and the male partner’s motivations for study participation (e.g., to please their wives vs. to help the researchers). In this context, recruiting couples in less stable relationships was difficult overall, but interestingly, women in unstable relationships appeared more motivated to recruit their male partners alone, rather than allowing the researchers to recruit more directly. By documenting and reporting the various recruitment strategies undertaken to enroll participants, the study team was able to better appreciate how each method introduced bias in their sample of dyads. Though the researchers went through considerable effort to explore potential bias in the recruitment of their sample, they were not able to report details on the male partners that chose not to participate, nor on the women that refused participation, because they relied on a sample of convenience.

The primary aim of this study was to describe two recruitment strategies used to enroll the spouses of services members into a large prospective study of military couples, and to assess the impact of recruitment strategy on the representativeness of the final study sample. The use of two different recruitment strategies provided an opportunity to assess selection bias [15,16,17] by comparing dyadic data collected with versus without service member referral. Given this unique opportunity, the study sought to address the following research questions (RQs):

• RQ1: What service member characteristics are associated with referring spouses?

• RQ2: Did recruitment strategy impact spouse response rates or moderate factors associated with response propensity?

• RQ3: Did recruitment strategy impact final spouse representation?

This study was facilitated by the availability of sociodemographic information for all of the military couples in the recruitment population.

Methods

Study population and procedures

The Millennium Cohort Family Study (Family Study), an ongoing probability-based cohort study, plans to follow 9872 US active duty and Reserve/National Guard service-affiliated married couples for 21 years [18]. The Family Study provides a unique opportunity to assess the impact of a dyadic recruitment strategy on study sample representation. The Family Study includes couple dyads comprising a military spouse who completed the Family Study questionnaire and a service member who participated in the larger Millennium Cohort Study [19,20,21]. Dyadic recruitment for the Family Study targeted spouses of Millennium Cohort Study participants with 2–5 years of military service who completed a questionnaire during the 2011–2013 survey cycle (n = 28,603 eligible service members). Female and married service members were oversampled to ensure adequate representation in the Family Study.

Newly enrolled married service members were asked to refer their spouses to the Family Study upon completion of the online Millennium Cohort Study questionnaire by providing contact information for their spouse (postal and email addresses.¹). Approximately one-third of spouses were referred, so the study team was able to invite them to participate immediately using both postal mail and email address information. In cases where contact information was not provided, but the service member did not specifically decline the request to contact his/her spouse, contact information obtained from Department of Defense (DoD) administrative records was used to invite these spouses by postal mail. In order to successfully engage spouses, considerable attention was given to designing each contact (both postal and email) for this study. The theoretical basis for designing each contact drew heavily from social exchange theory as presented by Dillman et al. [22]. The Family Study methods are described in more detail elsewhere [23].

Analytic subgroups

To investigate our RQs, three analytic subgroups were utilized: (RQ1) newly enrolled married service members who completed the 2011–2013 online Millennium Cohort Study questionnaire (n = 25,017); (RQ 2) spouses invited to complete the Family Study questionnaire either via service member referral or direct contact, excluding spouses of service members who responded by paper where no referral option was offered and spouses with insufficient contact information available from military records (n = 19,937); and (RQ3) spouse responders to the Family Study questionnaire, excluding spouses of service members who responded by paper (n = 8743). For RQ1, service member respondents were categorized into one of two recruitment groups: those who referred their spouse to the Family Study (Referring) and those who either refused to refer their spouse or submitted the survey without responding to the referral item (Refuse/Skip.²). For RQ2, spouses were categorized as responders and nonresponders to the Family Study survey. For RQ3, spouse respondents were categorized into two final status groups: those who were referred (Referred) and those who were contacted without a referral (Direct).

Measures

Demographic and military data

Sociodemographic and military data for service members were obtained from DoD electronic personnel files maintained by the Defense Manpower Data Center and included service member sex, age, race/ethnicity, service branch, service component, military pay grade, deployment dates, and number of dependents. Service member participants were categorized as deployers if they had at least one deployment in support of Operations Enduring Freedom, Iraqi Freedom, and New Dawn prior to their survey completion date; otherwise, they were classified as nondeployers. Deployers were further classified as deployed with and without combat, based on an affirmative self-report to any of the combat-like experiences on the Millennium Cohort Study questionnaire (e.g., witnessing death, trauma, prisoners of war, or refugees) or on the Post-Deployment Health Assessment (e.g., feeling in danger of being killed, being in direct combat where you discharged a weapon, or encountered dead bodies or people being killed or wounded) [24]. Additionally, service members self-reported education. Spouses self-reported sex, age, race/ethnicity, household income, employment status, prior/current military experience, education, and number of children in household.

Stress

The Patient Health Questionnaire [25] was used to capture common stressors experienced in the last 4 weeks by both service members and spouses, including: 1) financial strain represented by “financial problems or worries”; 2) relationship stress represented by “difficulties with husband/wife, partner/lover, or boyfriend/girlfriend”; 3) caregiver burden associated with “taking care of children, parents, or other family members”; and 4) lack of social support indicated by “having no one to turn to when you have a problem.” Reponses were provided on a 3-point scale (1 = Not bothered, 2 = Bothered a little, 3 = Bothered a lot); however, responses for each item were collapsed to create a binary variable that distinguished those who responded “not bothered” from those who responded “bothered a little or a lot.”

Stress of military life

Spouses were asked to report the average number of hours per week their service member worked during the past month, as well as the total number of days in the past year the service member was away from home, inclusive of deployments, training, and temporary duty assignments. In addition, spouses completed the Work–Family Conflict Scale [26] modified for military families (5 items from 1 = Strongly disagree to 5 = Strongly agree; α = .90). Spouses also rated nine military life experiences categorized into Deployment Stress (e.g., combat-related assignment), Injury Stress (e.g., caring for ill, injured, or disabled spouse), and Family Stress (e.g., military duties interfering with family time) based on a 4-point Likert scale ranging from 1 (Not stressful at all) to 4 (Very stressful), if they had experienced the event in the last 12 months. Events were assigned a score of 0 if spouses reported not experiencing the event in the last 12 months.

Marital quality and duration

Marital quality was reported by spouses on the Family Study questionnaire using four items from the Quality of Marriage Index [27], which were used previously in large-scale, epidemiological research with deploying military personnel [28]. Items were rated on a 5-point scale (from 1 = Strongly disagree to 5 = Strongly agree) and averaged to create an overall scale score ranging from 1 to 5. Spouses also self-reported length of marriage.

Individual health and adjustment

Overall physical and mental health were assessed for service members and spouses using the Medical Outcomes Study Short Form 36-Item Health Survey for Veterans [29, 30], which yields eight scales that combine to create mental component summary (MCS) and physical component summary (PCS) scores. PCS and MCS scores were categorized into three groups (highest 15th percentile, middle 70th percentile, or lowest 15th percentile) to approximate one standard deviation around the mean.

Statistical analyses

Descriptive statistics and bivariate analyses, including chi-squared tests for categorical variables and t-tests for continuous variables, were performed to identify service member and spouse characteristics associated with Family Study recruitment. In order to assess the adjusted association between service member characteristics and referral choice (RQ1), logistic regression was performed. Spouse response rate comparisons (RQ2) were assessed using chi-squared tests. To determine if recruitment group moderated characteristics associated with spousal participation, logistic regression was used to test for interactions with recruitment group (RQ2) in modeling response propensity [18]. Finally, to investigate whether there were significant demographic differences across recruitment groups represented in the final sample (RQ3), a series of chi-squared tests and t-tests were performed to examine differences in demographic, military, individual health and adjustment, marital, military life, and family variables. All analyses were weighted to account for nonresponse to the Millennium Cohort Study questionnaire. All analyses were conducted using SAS software, version 9.4 (SAS Institute, Inc., Cary, NC, USA).

Results

RQ1: What service member characteristics are associated with referring spouses?

Descriptive characteristics of married service member participants for the overall sample and by referral choice are presented in Table 1. Of the 25,017 married service members who completed the web survey, the majority was male, 34 years old or younger, identified as non-Hispanic White, and had less than a bachelor’s degree. More than half of service members reported having one or more children; that they were not bothered by their significant other, caring for others, financial stress, or having no one to turn to; and that they were in good mental and physical health. With regard to military characteristics, the majority of service members had deployed with combat and was enlisted, in the Army, and active duty.

Table 1 Characteristics of married Millennium Cohort Study participants for overall sample and by referral choice

Table 2 presents the logistic regression model examining the relationship of service member demographics, military characteristics, and individual health and adjustment with spouse referral. The unadjusted odds ratio for each variable is also presented to show bivariate contributions to spouse referral. When considered simultaneously in the fully adjusted model, service member demographic characteristics associated with referring spouses to the Family Study were male sex, bachelor’s degree or higher, more than one child, and feeling bothered by financial stress. Service members between 25 and 44 years of age and minorities were less likely to refer their spouse. Military characteristics revealed those deployed with combat, and Reservists or National Guardsmen were more likely to refer. Additionally, those who reported physical health (PCS) in the highest 15th percentile were significantly less likely to refer their spouse to the Family Study. Lastly, service members were more likely to refer their spouse if they were bothered by relationship difficulties and by caring for others.

Table 2 Unadjusted and adjusted ORs for service member characteristics predicting service member choice to refer (N = 23,319)

RQ2: Did recruitment strategy impact spouse response rates or moderate factors associated with response propensity?

For these analyses, there were 19,937 eligible spouses invited via referral (n = 8209) or direct contact (n = 11,728) to participate in the Family Study. Nearly half of the invited spouses (n = 8789, 44.1%) responded by completing or partially completing the Family Study questionnaire.³

Nearly two-thirds (n = 5331, 64.9%) of referred spouses responded, compared with less than one-third (n = 3458, 29.5%) of those contacted directly (χ² = 2462.72, p < .001). To determine the overall success of the referral recruitment strategy, service member referral rate (34%) along with spouse response rate (65%) must be considered. Thus, only 27% of eligible spouses participated using a referral strategy.

To determine if recruitment strategy differentially impacted the likelihood spouses would respond to the survey across demographic subgroups, we tested the interaction of recruitment group (Direct vs. Referred) with 15 service member characteristics (e.g., sex, age, race/ethnicity, children, combat) used to predict response propensity in a previous analysis [18]. Out of all the service member characteristics used to predict Family Study survey response, only the interaction term for children and recruitment group was marginally significant after Bonferroni alpha adjustment (p < .003; .05/15). Specifically, having children increased response for spouses contacted directly and slightly decreased response for spouses who were referred (p = .003).

RQ3: Did recruitment strategy impact final spouse representation?

A comparison of Family Study spouse responders across 24 different demographic, military, individual health and adjustment, marital, and family indicators revealed numerous significant differences between recruitment groups (Table 3). Using Bonferroni adjusted alpha levels of .002 per test (.05/24), chi-squared tests and t-tests revealed that Referred spouses compared with Direct spouses were more likely to be in the youngest age group (31.7% vs. 25.1%), have 3 or more children (12.4% vs. 8.2%), categorize themselves as homemakers (36.2% vs. 33.5%), and be bothered by financial problems (57.7% vs. 53.3%). With regard to service member military indicators, Referred spouses compared with Direct spouses were more likely to be married to service members with combat deployment experience (59.8% vs. 54.4%), Army affiliation (53.3% vs. 45%), and Reserve/National Guard status (21.9% vs. 18.6%). Additionally, Referred spouses reported service members were away from home due to military duties a greater number of months than Direct spouses (3.6 vs. 3.4). In addition, Referred spouses compared with Direct spouses were more likely to be married less than 2 years (19.4% vs. 14.7%) and report being bothered by difficulties with their significant other (37.2% vs. 32.9%).

Table 3 Characteristics of Millennium Cohort Family Study participants by recruitment status (N = 8743)

Discussion

Few large scale studies have recruited married couples, and among those studies that have, it is unclear whether or how spousal referrals were incorporated into their recruitment strategy [8,9,10,11], making it impossible to determine selection bias and representativeness of the study sample. The Family Study offers a unique opportunity to thoroughly examine the potential for bias when one member of a dyad is asked to refer the other by analyzing extensive data collected from military spouse dyads, including sociodemographic and psychosocial characteristics. For this study, we were able to: 1) examine service member characteristics associated with providing a referral compared with those who chose not to refer; 2) determine if spouse survey response was impacted by service member referral choice; and 3) assess whether responding spouses who were referred differed significantly from spouses who were contacted without a referral. To our knowledge, this is the first large scale study to provide a detailed assessment of the impact of a referral recruitment strategy compared with a direct contact recruitment strategy on the representativeness of the final dyadic study sample.

The results of our initial unadjusted analyses examining service member characteristics associated with providing a spouse referral (RQ1) showed that the majority of service member sociodemographic and military variables were significantly associated with referring spouses, along with various mental and physical health composite measures, and financial and social concerns. Similar to the Family Study nonresponse findings [18] and those of other survey research studies, providing a referral is related to sociodemographic characteristics strongly related to participating in survey research. In adjusted analyses, service member sex, age, race/ethnicity, education, and financial concerns were associated with service member referral. Indeed, the strongest predictor of referring one’s spouse to the Family Study was being male followed by having children, with a greater number of children increasing this association. In addition, we found several military factors to be predictive of referral choice, such that those who experienced a combat deployment were more likely to refer their spouse to the Family Study than those who never deployed or deployed without combat; service members in the Reserve/National Guard were more likely to refer than active duty service members; and those serving in the Air Force were the least likely to refer their spouses. Other significant predictors of referral choice were being bothered by caring for others and marital difficulties.

These differences, however, are quite small and likely rooted both in the typical demographic correlates of survey response (e.g., sex and socioeconomic level) and unique motivations to support the objectives of this study (e.g., concerns regarding the impact of military life stress on families). If spousal referrals are similar to physician referrals, then referring spouses may have a greater interest in the research topic than non-referring spouses [31, 32]. That is, the systematic differences associated with service member referral choice seem to reflect a perception of spouse “fit” with the research topic, such that male service members with children, who had experienced a combat deployment, and who may be having difficulties at home see themselves and their spouses as suffering from an “ailment” related to the research topic [33,34,35]. Female service members may not think their male spouses “fit” with the research topic, especially if the spouse is also a service member or they do not have children.

Our examination of spouse response by referral status (RQ2) revealed that referred spouses were significantly more likely to respond to the Family Study survey than those who were not referred. This difference was likely the partial result of two known methodological differences in recruitment strategy. Spouses who were referred to the study with complete contact information were invited to participate in the Family Study immediately by email and within 2 weeks by postal mail, whereas there was a delay for spouses contacted without a referral. In addition, Referred spouses were contacted multiple times by email to complete the questionnaire and were provided an email embedded link to the web survey, whereas Direct spouses were only invited by postal mail to complete the web survey.⁴

Email augmentation has been shown to ease the response task and garner greater attention to the response request, resulting in improved response rates [22].

In addition to the methodological differences, our results indicate that service members are more likely to refer spouses who may have an interest in the study topic based on their perception of spouse “fit” with a survey of military families. That is, service members may have been more likely to refer their spouses if they thought their spouse would respond. Indeed, spouses may share this perception of survey fit—increasing the likelihood of interest in the study topic—and of survey response [22].

Although Referred spouses were significantly more likely to respond than Direct spouses, the success rate of using a referral strategy was less than when recruiting spouses directly. That is, the direct recruitment method provided a much greater number of spouses to be contacted than the referral recruitment method, so even though the response rate was lower for the Direct group than the Referred group, the percentage of eligible spouses responding using the Direct method was higher than when relying on referral. In other words, inviting spouses without a referral was more successful than inviting only spouses who were referred.

Lastly, we used a nonresponse model developed for the entire Family Study population to look for interactions between 15 response propensity characteristics and recruitment group on spouse survey response. We found that for all but one interaction, recruitment group did not significantly modify the relationship between nonresponse characteristics of the spouses invited to the Family Study and survey response. Only the interaction term for children and recruitment group was marginally significant, suggesting that having children slightly decreased the likelihood of response for the Referred group and slightly increased the likelihood of response for the Direct group. This suggests that recruitment strategy, for the most part, had little impact on spouse response characteristics.

A final comparison of Family Study participants (RQ3) revealed numerous differences between recruitment groups that closely mirrored the service member characteristics associated with referring spouses. That is, Referred spouses compared with Direct spouses were younger, had children, and reported being bothered by financial problems, marital problems, and caring for others. Likewise, the military characteristics of Referred spouses reflected those of Referring service members (e.g., combat deployment, Army, Reserve/National Guard). In addition, Referred spouses were more likely to describe their employment status as homemaker, to be married less than 2 years, and to report their service member spent more months away from home compared with Direct spouses. Again, these differences appear to reflect perceptions of “fit” with the research topic. We suspect these characteristics may indicate more identification with the stereotypes for military spouses, as well as greater interest in the focus and objectives of the study as a motivation for referral and participation. Furthermore, this also indicates that the total picture of military operational stress, rather than the difficulties of deployments alone, are perceived as relevant and should continue to be a focus in the Family Study as well as other research on military families.

Conclusion

Overall results suggest that minimal bias was introduced into the Family Study by using a referral recruitment methodology. Systematic differences in group membership appeared to be driven by service members referring spouses based on perceptions of “fit” with the research topic, which were further augmented by the increased response rate of spouses mirroring those characteristics. These group differences accounted for only a small proportion of variability in final group membership and provide confidence that minimal bias was introduced by using referral recruitment.