Study design and participants
We performed a secondary analysis of the Veterans Victory Over Tobacco Study (VICTORY), a pragmatic trial of a proactive outreach, population-based intervention for tobacco treatment. A full report of the methods and the primary outcome results of the original randomized trial have previously been published [15]. In brief, subjects were recruited from four VA sites, selected to be nationally representative. Sites included: G.V. (Sonny) Montgomery VA Medical Center (Jackson, MS), James A. Haley VA Medical Center (Tampa, FL), Minneapolis VA Medical Center (Minneapolis, MN), and New York Harbor VA Medical Center (New York, NY). 6400 participants were enrolled from October 2009 to September 2010 until we reached the number of participants necessary to detect a 2% difference in population level cessation outcomes, and follow-up was completed in November 2011. Subjects were randomly selected Veterans aged 18 to 80 years who were identified as likely current smokers. As part of the pragmatic nature of the trial, subjects were screened for inclusion via a HF indicating tobacco use during a primary care visit within the prior 3 months. Smoking status was then confirmed by study personnel prior to enrollment. Exclusion criteria were minimal (ICD-9 diagnosis of dementia or severe persistent mental illness, 10 or more mental health clinic visits in the prior year, no valid contact information). Inclusion/exclusion criteria were applied using the VA HER.
In order to maintain balance within site, participants were randomized 1:1 to usual care versus the proactive outreach intervention, clustered within each site, and were enrolled regardless of current intention to quit smoking. Participants in the usual care arm could access any tobacco treatments recommended by their provider, including services through the VA or the state quitline. The proactive arm received both 1) proactive outreach (mailed invitation materials followed by telephone outreach) followed by 2) the choice of telephone or in-person cessation counselling. Phone counseling was provided by trained counselors located at the Minneapolis VA. Subjects who chose in-person counseling were connected with local VA cessation services, and all subjects were able to receive medications through their VA provider, which was facilitated by the study team. The project was approved by the Minneapolis VA Institutional Review Board (IRB) and the IRB of each study site and all studies were conducted in accordance with the ethical principles outlined in the Department of Veterans Affairs Good Clinical Practices. Data was stored according to the VA requirements for encryption, with access to identifiable data limited to the minimum number of required study personnel.
Data collection and measures
Data was obtained via patient survey and the VA EHR. Survey data collection occurred by mail at baseline and at 12 months after randomization. Survey mailing staff were blind to the participants’ group assignment at the time of mailing. Smoking status was measured using self-reported 6-month prolonged smoking abstinence. The survey was developed in accordance with the recommendations for assessing smoking abstinence recommended by the Society for Research on Nicotine and Tobacco [16]. The study definition allowed for some lapses in cigarette smoking. Patients were categorized as not achieving 6-month prolonged abstinence if they indicated smoking in the past 30 days, had smoked at least once on seven consecutive days out of the past 6 months, or smoked at least once on two consecutive weekends. All analyses used self-reported 6-month prolonged abstinence as the conventional gold standard.
Administrative data, including demographics, clinical characteristics, and smoking status were obtained from the VA National Patient Care Databases. Demographic variables included age, gender and race. For individuals who had missing race on the baseline survey, race was filled in from administrative data in the EHR. Clinical characteristics were obtained from the EHR. We defined chronic lower respiratory disease (chronic bronchitis, chronic airway obstruction, emphysema, asthma, and bronchiectasis), mental illness (depression, post-traumatic stress disorder, anxiety, substance use disorder, serious mental illness), and cardiac disease using ICD-9 codes. We also used ICD-9 codes to calculate the Charlson comorbidity index, a validated administrative measure of the burden of disease, with higher scores indicating a greater burden [17].
We also measured smoking status using HFs. We extracted all tobacco-related HFs within 120 days before or after the 12-month follow-up survey mailing, therefore spanning a maximum time period of 240 days surrounding the date of mailing. HFs are generated when a member of the health care team enters a patient’s response into a specially templated form. These entries tag the note with a tobacco HF that, unlike a text entry in a progress note, is easily retrievable by electronic query. Nurses may be prompted to enter tobacco use information annually, at the time of a hospitalization, or change in clinical status.
All the resulting HFs were reviewed by the study team to assign the follow-up smoking status (current smoker or quitter). For example, a patient was identified as an ongoing smoker if the HF label contained information indicating: ‘CURRENT SMOKER’, ‘TOBACCO CURRENT USER’, ‘REFUSED SMOKING CESSATION’, ‘TOBACCO USER’, or similar. A patient was identified as a quitter if their follow-up tobacco status indicated ‘QUIT TOBACCO IN THE LAST 12 MONTHS’, ‘NON TOBACCO USER - QUIT IN PAST YEAR,’ ‘CURRENT NON-TOBACCO USER’ or similar. See Additional file 1 for complete definitions of tobacco use statuses. Patients whose HFs indicated use of a different form of tobacco, such as chewing tobacco, but not cigarettes were characterized as nonsmokers.
Statistical analysis
Participants were grouped by source(s) of their follow-up smoking status and various baseline characteristics were compared by these four resulting groups (survey only, HF only, both, and neither) by using Pearson Chi-square tests for categorical characteristics or by Kruskal-Wallis rank sum tests for continuous characteristics to assess if there were any group differences in baseline characteristics.
Among participants with follow-up smoking status available from both data sources, agreement between 6-month prolonged abstinence from smoking status by source (survey versus HF) was calculated overall, then also by various date ranges, site, and subgroups of clinical interest. Positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, and the Kappa statistic with 95% confidence interval were also calculated. PPV and sensitivity was calculated in reference to correctly identifying a quitter, and NPV and specificity was calculated in reference to correctly identifying an ongoing smoker. A Kappa value of 0.4–0.6 was considered moderately good agreement, and a Kappa of > 0.6 was considered good agreement. McNemar test was used to test if the marginal proportions of quitters from either source are equivalent or not.
To estimate the standard errors of PPV, NPV, sensitivity, and specificity we used a first order Taylor expansion approximation for the variance of the ratio of random variables, since the denominator of these two probabilities is also random [18]. To see if treatment had any effect on the main results, we repeated our main analyses within each treatment arm separately. The analysis for this paper was conducted by using SAS/STAT software, Version 9.2, by two members of the statistical core of the Center for Chronic Disease Outcomes Research (See author byline).