Parent study population
This incentive study was an ancillary study of non-respondents from a larger face-to-face survey of veterans. The study population for the larger main study included all primary care patients at four Veterans Health Administration (VHA) medical centers (Minneapolis, MN; West Los Angeles, CA; Portland, OR; Durham, NC) who were scheduled to have at least one primary care visit during the study recruitment period (June, 2004 through May, 2005) and who did not suffer from a severe cognitive disorder (i.e., Alzheimer's disease, severe dementia, schizophrenia) or blindness, as determined from an initial review of medical records. Invitations to participate in the main study, which included a face-to-face interview at the medical center to assess the patient's health literacy skills and an offer of a $25 cash incentive for completing the interview, were mailed to randomly selected patients at each site and then participants were recruited by phone. Study recruiters telephoned each potential participant to determine their willingness to participate approximately 10 days after the mailed invitations were sent. Six attempts were made to reach participants at different times of day. Patients were classified into willing participants, hard refusers (e.g., did not want anything to do with research), soft refusers (e.g., could not participate because of logistical reasons), and those whom we could not reach by phone or mail. Those who were reached and willing to participate booked a one-hour research appointment, usually on the same day as their scheduled primary care appointment[20]. Institutional Review Boards from the each study site (Minneapolis, MN; West Los Angeles, CA; Portland, OR; Durham, NC) approved the study protocol.
Ancillary study population
In order to assess the effect of non-participation on prevalence estimates of poor health literacy, eligible patients from the parent study who could not be reached, did not attend their scheduled research appointment, or refused because of transportation, scheduling difficulties, or other conflicts were mailed a one page, plain language, ancillary survey designed to characterize non-responders. This group of parent study non-respondents, as outlined in Figure 1, comprises the sample for this study.
Data and measurement
Data from the ancillary survey and medical record and administrative sources were used as independent variables in this investigation. Sex, age (< 50, 50-75, > 75), urban/rural residence (determined from U.S. census data), comorbidity history and mental health diagnoses were extracted from VHA administrative and medical record data; therefore, data were available for ancillary survey respondents and non-respondents. Comorbidities were summarized using the Charlson Comorbidity Index score[21] and a measure of mental health diagnoses which categorized individuals into one of three groups: (1) no mental health diagnoses, (2) at least one psychiatric (ICD-9 codes 290-302 and 306-311) or substance abuse related (ICD-9 codes 303-305) diagnosis, or (3) dual diagnosis (psychiatric and substance abuse). Measures of mental health diagnoses were included because they could conceivably affect the accuracy of survey response and are not captured in the Charlson Comorbidity Index. Ancillary survey questions included 4 self-reported health literacy questions[22], marital status (recoded into married versus unmarried), race (white, African American, other), education (≤ high school, some college, ≥ college graduate), employment status (employed versus not employed) and income (recoded into ≥ $20,000, $20-40,000, > $40,000).
Ancillary survey administration
The ancillary survey was administered by a university-affiliated survey center not associated with the parent study or its investigators. In three of the medical center sites, patients were randomly assigned to either a $2 or $5 prepaid cash incentive condition, and the data from these three sites were used for this paper. The fourth site did not participate in the incentive experiment. Survey packets were mailed using first-class postage to 1,328 patients. Participants were assigned random numbers between 0 and 1 and then based on the number (< 0.5 or > 0.5), participants split into two equal groups. Randomization was performed blindly, with no information about individual cases to affect the distribution. The first survey packet included the cash incentive, a cover letter, a pre-addressed postage-paid business envelope and the 10-item questionnaire. A reminder postcard was mailed approximately one week after the first packet. A second questionnaire packet without an incentive was mailed to any subject who did not return a blank or completed survey within 3-4 weeks of the first mailing. Surveys returned before the mailing of the second questionnaire were categorized "first mailing respondents." Surveys received after this period were categorized "second mailing respondents."
Analysis
All analysis was completed using SAS version 9.1. To determine the success of the randomization, contrasts of the respondent's demographic and health characteristics by their incentive condition were first compared using chi-square tests. Significant differences found from these results were later controlled for in logistic regressions. Response rates were calculated using American Association of Public Opinion Research (AAPOR) RR1 criteria[23]. Differences in response rates by incentive group were compared by demographic and health factors. Cumulative response rates by incentive group ($2 or $5) were then compared for the "first mailing participants" and then for the "second mailing participants." Logistic regression was then used to calculate the odds of responding to the survey with a $5 versus a $2 incentive, controlling for variables that were not successfully randomized (sex, rural/urban status, and site), as well as incentive. Because other demographic and health characteristics were not significantly different in bivariate analyses, these variables were not included in the regression analyses used to calculate odds ratios.
The cost of each survey packet included materials (1 page survey, outgoing and return envelopes), postage (outgoing and return) and labor. Labor costs per survey were calculated by multiplying the hourly rate, including benefits, in the year the survey was fielded (2005) by the time that employees and supervisors spent stamping, sealing and stuffing packets as well as logging returns and data entry of survey results, divided by the number of surveys sent. Time spent processing returned surveys was calculated for the total response to each mailing, without regard to incentive status. The cost of the reminder postcard includes printing, labor and outgoing postage. The reminder postcard was sent to all recipients of the first mailing and is included in the total cost of that mailing. The cost per completed survey was determined for each round of mailing by dividing the total cost of the mailings by the number of completed surveys returned. Indirect costs were not included.
In order to assess whether ancillary survey respondents were representative of all those to whom the survey was sent, demographic and health status of respondents for each incentive condition and mailing were compared to the entire ancillary study sample.