We explored compliance in five studies that had used frequent SMS to collect data over a fairly long time. Overall compliance was somewhat different between the studies, ranging from 98.2% (MC Chiro) to 88.1% (PARA). However, in the great scheme of clinical research, an overall compliance of around 90% is quite remarkable. Further, a compliance around 70% at the end of a study period of 52 weeks (the lowest compliance of the Weekly studies) is in line with previous studies using SMS to collect data [7, 9], and must be considered a high figure in comparison with other means of collecting data [22, 23].
The included populations were intentionally different, in order to get a variety of factors where compliance with the SMS methodology could be explored. Four studies with working populations were included: MC Chiro, SCIP Falls, Work-Up and SPA, and two populations consisted of patients with chronic conditions, SCIP Falls and PARA. Therefore, not surprisingly, the subjects in MC Chiro, although clearly in pain when consulting for care, consisted of healthy subjects without much sick-leave. The subjects in SCIP Falls, who all lived with spinal cord injuries, reported the poorest quality of life according to the EuroQol 5 dimensions (EQ-5D) index.
Some gender differences between the populations were observed, some of which could be explained by the nature of the inclusion: PARA had the highest proportion of females, as RA is a condition affecting more females (gender ratio females/males of 80/20). In SPA, the population was also largely female, as the study was set among health care workers, traditionally female-dominated professions. In SCIP Falls, however, the participants were largely male [24]. This is also expected as the gender ratio (females/males) of SCI in Norway and Sweden is 25/75. In previous studies using the SMS technology, gender differences in compliance were minimal [6, 9].
We found that compliance was influenced to some degree by individual factors, but the factor that remained significant in the final model was “study”, i.e. the individual factors did not explain the difference in compliance as much as the study itself. We therefore need to look at the specifics of the studies to answer the questions of this paper.
Question 1: Is compliance associated with study-specific factors? There were some indications that age and gender did influence compliance (Fig. 3), but not as much as “study”. This “study effect” may reflect the condition of the subjects. Throughout, PARA experienced lower compliance than the other studies, their subjects suffering from a chronic condition, Rheumatiod Arthritis. The best compliance was found in MC Chiro, were subjects had recurrent and persistent LBP. Studying the health indicators of the subjects, participants in MC Chiro and Work-Up reported far more pain and poorer health compared to PARA participants. However, in chronic conditions such as RA, it may well be that individuals adapt to pain over the years, implying that it is not regarded as bad as if it was acute. Thus, it is difficult to conclude if condition may be a factor explaining compliance.
It is also possible that the severity of the condition explains the “study effect”, as highlighted in a previous study [9]. This was tested by using studies where categorization due to pain was possible (MC Chiro, Work-Up and PARA), but compliance was not found to be consistently different among groups of respondents with different levels in of pain intensity (Fig. 3, Tables 3, 4 and 5). Lastly, these three pain conditions are fluctuating, and it is possible that the “study effect” would be explained by changes in the condition. We calculated initial pain intensity development (from study start to 8 weeks) but did not find differences in compliance between those who improved, stayed the same or deteriorated (data not shown).
The “study effect” may be a result of the condition but does not seem to be attributable to severity or development of this condition, nor other available population factors. It may be due to other unmeasured differences in these populations, as we could only make comparisons across common variables.
Question 2: Is compliance associated with the SMS- methodology itself? In total 8 different questions (1 each from MC Chiro, SCIP Falls and SPA, 2 from PARA and 3 from Work-Up) were asked across the studies, all to some extent “sensitive”. However, in PARA, the SMS questions were about compliance with the intervention, i.e. physical activity. These questions were related to a socially desired behavior that was supposed to increase during the study period, while the other studies mainly measured disease-related symptoms. Failure in behavior change might certainly have been the case for lower compliance in PARA, while there is no expected failure related to the reporting of symptoms as in MC Chiro, SCIP Falls, Work-Up and SPA.
Questions have been raised concerning the influence of the technology itself on the outcome, i.e. does frequent questions about pain lead to more pain [25, 26], or is it stressful to answer frequent questions about stress [27]? However, that does not seem to be the case. In PARA, it could be proposed that the weekly SMS would act like a prompt to be more physically active, as hypothesized by others [28], but whether this prompting actually influences compliance is unknown.
Three studies were using weekly questions, one study was using questions every 2 weeks (SCIP Falls) and one (SPA) used a regimen of collecting data for 13 weeks, pausing for 12 weeks and then collecting data again for 26 weeks. If frequent questions were perceived as a burden, SCIP Falls and possibly SPA should show the highest compliance, which was not the case. It was, however, clear that holidays rendered somewhat lower compliance, as this was observed even in MC Chiro. The longest study (PARA) admittedly had the lowest compliance, but examining only the first year of PARA, compliance was lower already. Thus, the number and frequency of questions asked, as well as the study duration, did not explain differences in compliance between the studies.
In these five studies, all PI’s were given the same instructions how to oversee the data collection, send reminders and call non-compliers. It is possible that compliance with the rigorous execution of these management procedures was explaining the “study effect”, that it was a “management effect”. Indeed, early compliance was found to influence “stamina”, i.e. compliance throughout the study. As found in a previous scrutiny of the SMS method, it is important to motivate the non-responders early on [9]. In SCIP Falls, the participants were called by the investigators if answering “yes” (= I did fall), and this attention may also have contributed to the high compliance.