This article has Open Peer Review reports available.
Should we embed randomized controlled trials within action research: arguing from a case study of telemonitoring
© The Author(s). 2016
Received: 1 December 2015
Accepted: 28 May 2016
Published: 8 June 2016
Action research (AR) and randomized controlled trials (RCTs) are usually considered to be theoretically and practically incompatible. However, we argue that their respective strengths and weaknesses can be complementary. We illustrate our argument from a recent study assessing the effect of telemonitoring on health-related quality of life, self-care, hospital use, costs and the experiences of patients, informal carers and health care professionals in two urban hospital services and one remote rural primary care service in New Zealand.
Data came from authors’ observations and field notes of discussions with three groups: the healthcare providers and healthcare consumers who participated in the research, and a group of 17 researchers and collaborators. The consumers had heart failure (Site A, urban), airways disease (Site B, urban), and diabetes (Site C, rural). The research ran from 2008 (project inception) until 2012 (project close-off). Researchers came from a wide range of disciplines. Both RCT and AR methods were recognised from early in the process but often worked in parallel rather than together. In retrospect, we have mapped our observed research processes to the AR cycle characteristics (creation of communicative space, democracy and participation, iterative learning and improvement, emergence, and accommodation of different ways of knowing).
We describe the context, conduct and outcomes of the telemonitoring trial, framing the overall process in the language of AR. Although not fully articulated at the time, AR processes made the RCT sensitive to important context, e.g. clinical processes. They resulted in substantive changes to the design and conduct of the RCT, and to interpretation and uptake of findings, e.g. a simpler technology procurement process emerged. Creating a communicative space enabled co-design between the researcher group and collaborators from the provider participant group, and a stronger RCT design.
It appears possible to enhance the utility of RCTs by explicitly embedding them in an AR framework to shape stronger RCT design. The AR process and characteristics may enable researchers to evaluate telehealth while enhancing rather than compromising the quality of an RCT, where research results are returned to practice as part of the research process.
Australian New Zealand Clinical Trials Registry, reference ACTRN12610000269033.
Good clinical decisions require evidence on more than just effectiveness . Clinicians, consumers, and health systems, need evidence about processes, social context, patient engagement, equity, and health literacy; such factors are typically and explicitly eliminated from Randomised Controlled Trial (RCT) designs as sources of bias and confounding. Interventions whose effectiveness may vary with context, such as information and communication technologies (ICT), and many other health services interventions, challenge the ability to reduce bias and even the wisdom of attempting to do so.
Strengths and weaknesses of RCTs and Action Research
Epistemology (positivist, critical, interpretivist)
Ways of knowing
Aim and design
Aims at improvement
Aims to measure effectiveness of a clinical intervention
Yes, among other things
Co-design of research plan, involving participants
Participatory and democratic
Controls for bias and confounding factors
Accounts for and investigates context, social processes, patient engagement, equity
Measures context-dependent interventions and interactions
Creates communicative space
At design phase
Intervention improvement via cyclical iterations
Design adjusted concurrent to emerging findings
Derives data and results from practice of reflexivity
Emergence (new, unexpected/expected knowledge)
Yes, as incidental findings, unintended consequences
Yes, as emergent findings specifically sought
Results in immediate multidimensional change
Results in later change in clinical practice
The purpose of this article is to examine the use of AR cycles and characteristics as a frame for designing and executing a telemonitoring RCT. The case we use in our argument aimed to assess the effect of telemonitoring on health-related quality of life, self-care, hospital use, costs and the experiences of patients, informal carers, and health care professionals. We present background literature, a telemonitoring RCT as the case study, and a critical analysis of our AR experience related to the case study.
Acquiring evidence for practice
RCTs are not enough for acquiring evidence and applying it in practice. It can take up to 17 years for research from RCTs to become everyday clinical practice, because of repetitive research projects, and difficulties in identifying relevant research for practice . Research into telehealth is not compatible with that timetable, as technology is constantly changing as refinements and innovations emerge [7, 8]. Kaldoudi, Chatzopoulou, and Vargemezis  argue that RCTs alone may not be enough for telehealth because the intervention is more than the technology. Furthermore, the intervention can influence and be influenced by uncontrollable confounding factors, e.g., local context of the patient, the organisation, and healthcare processes. Bias cannot be controlled because telehealth RCTs cannot be blinded - it is obvious to participants that groups with and without the technology are being compared .
Telehealth consists of complex interventions. The UK Medical Research Council’s (MRC) evaluation framework defines complex interventions as having a number of intervention components, and associated behaviours, groups, organisational levels, intervention flexibility, and number and variability of outcomes . Telehealth research is usually complex, involving patients doing self-care, people being cared for at a distance with complex processes that are not bounded by the tyranny of geography or time, and extend beyond the patient-clinician relationship and a single organisation, to include multiple members of the multi-disciplinary team and the technology vendor.
Complex interventions require complex evaluations. Different methods can and should be used to evaluate interventions, and RCTs have a place, but do not sufficiently evaluate complex interventions, especially those that involve ICT. Lewin, Glenton and Oxman  conducted a systematic literature review of RCTs that included qualitative research, revealing that there is usually no overt link between the qualitative report and the RCT. The qualitative data is usually reported in the form of feasibility studies prior to an RCT. Campbell, Fitzpatrick, Haines, et al.  present a design for complex RCTs that includes a staged approach and collecting qualitative data, but does not address the difficulties associated with lack of blinding, and the rapid development of technologies. Mohr, Cheung, Schueller, et al.  propose a rapid cyclic form of RCT, but their method is limited to rapidly developing mobile technologies with high levels of sign-up and attrition. Murray, Teweek, Pope, et al.  propose that normalisation (i.e. an intervention becomes a normal part of everyday work) should be the goal of intervention evaluations such as RCTs. Their Normalisation Process Theory provides a framework for intervention designers to focus on implementation, such as context and change impact before a trial begins. Although they advocate for co-design, implementation considerations, and reflexivity (which are also AR features), once the trial begins these activities are set aside until the trial’s results are available. The AR feature of returning the results to the participants as part of the research is lacking.
Greenhalgh, Russel, Ashcroft, et al.  argue that ehealth evaluations require more than experimental cause and effect measurement because of the richness of what can be learned from the particular in complex situations. The MRC evaluation framework  has potential for thick description that can arise from AR. The MRC framework is a cycle consisting of feasibility study, evaluation of effectiveness and change process, implementation (implementation, surveillance and follow up), and development (theory, modelling and evidence base development). There are opportunities for gathering the rich data that falls around RCTs, but without the properties and processes of Action Research, these opportunities to apply learning from each step of the cycle and effect change can be lost.
Since telehealth interventions are complex, and although RCTs are increasingly built to accommodate complexity, the evaluation of these interventions requires a fuller, richer and therefore thick description of the intervention and its effects. The acquisition and use of evidence should be part of the research process, built in from the first design step, and fed back to practitioners at every stage of the research. We propose Action Research as a framework to address the limitations of RCTs in evaluating telehealth interventions.
Action research and the RCT
‘a participatory, democratic process concerned with developing practical knowledge in the pursuit of worthwhile human purposes….It seems to bring together action and reflection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the flourishing of individual persons and their communities.’ 
AR is usually described as a cyclic research process in which participants plan, act, reflect on the results of their actions, and modify the plan for the next cycle, repeated until they achieve their goal . Research and action are conducted together, the research informing and informed by each cycle of action. All this is done in a mutually agreed upon ethical framework . AR involves participants in defining a problem and designing and testing potential solutions, e.g. interventions, service improvements, trial methods. This is more than co-design and could provide useful input to build a strong RCT design that results in successful intervention implementation and return of research results to the involved community.
Create a communicative space at the beginning of a project. Participants are identified and agree to contribute to the project’s direction and content.
Iterative improvement is core to an AR project. It sets the tone for the change desired by the participants.
Emergence occurs as a result of informed action – as research produces knowledge and shared understandings, new (expected and unexpected) knowledge can be incorporated into subsequent iterations.
There are different ways of knowing, many perspectives. Each way of knowing is valid.
Democracy and participation are the foundation of AR.
The case: telemonitoring for long term condition management
We designed an RCT aimed at assessing the effects of telemonitoring on health-related quality of life, self-care, hospital use, costs and the experiences of patients, informal carers and health care professionals . The RCT intervention consisted of telemonitoring of vital signs measured by participating patients plus patient answers to daily questions about symptoms. Data were collected in an information system, which the research nurses assessed and responded to according to an established clinical protocol as described by Kenealy et al. .
Participating patients in the intervention arm were given the equipment and trained on how to use it. The control patients continued to receive ‘usual care’ by their clinicians. There were three sites. Site A was urban and the patients had heart failure. Site B was also urban and the patients had Chronic Obstructive Airways Disease (COPD). Site A and B patients were recruited from hospital specialists. Site C was rural, and the patients were recruited from general practitioners, regardless of diagnosis. These patients were mostly diabetic.
RCT participants grouped as healthcare providers and consumers
Site A (heart failure)
Site B (COPD)
Site C (diabetes)
3 respiratory physicians
2 general practitioners
3 heart failure nurse specialists
2 respiratory nurse specialists
1 practice nurse
1 hospital manager
1 respiratory nurse specialist with prescribing rights
1 rural health nurse
1 hospital manager
1 kaiawhina (community health worker)
49 telemonitoring hospital consumers with congestive heart failure
24 telemonitoring hospital consumers with chronic obstructive pulmonary disease
25 telemonitoring primary care consumers with diabetes and multiple long term conditions
49 usual care hospital consumers with congestive heart failure
24 usual care hospital consumers with chronic obstructive pulmonary disease
0 usual care consumers
During the planning phase of the RCT a small sub-group of the researchers negotiated the use of AR to frame this complex RCT. The goal was to use stages and sub-stages of the RCT plus the AR characteristics to enhance the implementation of the intervention and return the results of the study as soon as possible into practice. While it was considered good practice by the research team to negotiate the intervention design, the AR advocates were not able to negotiate the inclusion of potential RCT participants (i.e. specifically the patients/consumers) to co-define the research problem and participate in the co-design of the intervention. At the time it appeared appropriate to exclude potential RCT participants (consumers) because of the requirement to eliminate confounding factors and bias – inclusion was considered to undermine the RCT purpose, design and results. We did not manage to negotiate some of the funding to be focussed on the AR aspects of the fully funded RCT process and activities.
The academic who initiated the RCT project recruited clinicians, managers and community leaders from his extensive network to participate as co-researchers in the initial stages. There was a collaborator from each site – two were specialist clinicians and the third site was represented by a manager. Once the design had been established for the purpose of funding applications, these people became participants in the different sites that they belonged to, and are represented as such in Table 2. These three people remained in the AR team as well.
The research team was large and multidisciplinary. There were 13 academics from two faculties (Medical and Health Sciences, and Business), and covered the fields of medicine, nursing, health informatics, general practice, population health, and accounting and economics. A separate project team was established to support the funding applications, procurement of the telemonitoring equipment and information management system, and provide budget and project management services. For the purpose of this article, the AR team consisted of the academics, project manager, and site collaborators, totalling 17. The vendor supplied and supported the technology but did not become a member of the AR team as they were contracted to provide a service. The research was described and funded as an RCT, not an AR project, although we negotiated AR as a frame.
Research timeline and activities
Sept 2010 – Aug 2011
Ended Jun 2012
Identify and define the research problem
Site A: Sep 2010 – Aug 2011
Site B: Dec 2010 – Apr 2011
Site C: Sep 2010 – Feb 2011
Design research protocol
Refine and finalise RCT protocol
Enrol patient and clinician participants
Retrieve monitoring equipment.
Apply for funding
Confirm research team
Recruit research assistants
Reports in journals
Confirm data gathering tools
Refine RCT protocol
Use suite of data gathering tools
Disseminate results to participating District Health Boards
Confirm recruitment processes
Refine clinical pathways
Confirm patient and clinician participant recruitment process
In summary, three groups formed. The researcher group consisted of academics, some of whom were initially also clinicians from the different sites. Once the design was clarified, clinicians from Sites A and B shifted to the healthcare provider participant group, and the manager from Site C joined the researchers. The healthcare provider participant group was considered to be participants in that that they provided clinical services associated with the telemonitoring. They recruited patients, who made up the third group of consumer participants.
AR as a frame
As the overall research project progressed, and sub-cycles of the intervention were implemented, the degrees of distance between the research team and the research nurses executing the intervention increased, resulting in lost opportunities for AR characteristics and cycles to be applied. Since the AR aspect of the project was not funded, the discipline of reflexivity, documenting observations and lessons learned, and the contributions of participants, were not built into the research design or followed up. As the project progressed the commitment to AR reduced as the commitment to the RCT increased.
Meetings commenced in the first cycle (starting in 2008) and continued until after the study finished in June 2012. Reflection diaries  were not included in our project, although they are standard practice in AR. During this time the emphasis progressively shifted from planning the RCT to interpreting data, planning for the dissemination of findings, and future implementation of telehealth in New Zealand.
AR characteristics to co-design an RCT
Creating a communicative space to achieve co-design
Creating a communicative space within the research team occurred during the funding and ethics activities, and was sustained throughout the project in different ways, as per the objectives in Table 3. Funding and ethics approval applications required discussions and collaborative decision making and writing. Once funding had been approved a workshop was held to further develop relationships with a broad range of healthcare service leaders, academics and others who wanted to participate. This workshop attracted attendees from several sites all over New Zealand; not all attendees were able to commit their service to the project but wanted to participate in its planning.
Once the communicative space had been created the research and healthcare provider groups finalised which services would participate and under what circumstances, e.g., one urban site (Site A) wanted to include patients with heart failure, while a clinician from another urban site offered his COPD service (Site B). In contrast, the participants from the rural site were from a remote community, Māori, and had diabetes (Site C). Terms of engagement for Site C followed a collaborative and participative protocol that involved community elders, and a review of the research protocol prior to formal engagement in the project. This process leveraged the democracy and participation characteristic of AR by means of Kaupapa Māori . Smith argues that “Kaupapa Māori is a 'local' theoretical positioning which is the modality through which the emancipatory goal of critical theory, in a specific historical, political and social context, is practised” . Kaupapa Māori research reflects a Māori world view privileging Māori values in the interests of developing a research framework that is culturally safe. Kaupapa Maori research can strengthen community relationships, particularly when the research is driven by those who live in the community of interest. As Maori seek to be self-determining, Smith argues that such research allows for solutions and interventions that can change and improve lived realities.
Different ways of knowing, many perspectives
Democracy and participation
Democracy and participation refers only to the research/AR group as described above. The researchers had different interests in the RCT and were asking different questions, e.g. consumer perspectives, technological aspects, costs and economics, management, clinical outcomes. Each researcher’s contribution was valued.
Constant iterative improvement
Constant iterative improvement was achieved by using AR cycles (Figs. 1 and 2). The project manager used the typical project management PDSA cycle (plan, do, study, act) , to ensure that this complex RCT was appropriately executed.
We were obliged to use the New Zealand government’s procurement process to secure a vendor for the telemonitoring equipment. This process was drawn out and complex, and one of the incidental outcomes of the research process was a contribution to the development of a simpler process, ‘active procurement’ . The development of ‘active procurement’ could be attributed to the AR reflexivity that was being practiced in the early stage of the project. Such incidental outcomes (emergence) are common with AR .
Emergence is to be expected in AR. The cyclical constant improvement approach enabled us to look for consequences and address any adverse events early. One should expect unintended consequences to emerge in RCTs, and more so in health information technology research . We learned that the technology, despite being among the best available internationally at the time of selection, failed in a number of ways, e.g. lost data. Data quality can be compromised when patients are required to enter data themselves, rather than allow the system to automatically collect and transfer vital measurements. New processes of care emerged for remote patients (Site C) who otherwise would not have had access to service, but problems associated with distance and the need for additional support were also identified. An RCT focussed solely on the intervention may not have given sufficient space to identify these issues and address them and learn from action taken. Sites A and B started their version of the RCT at a different time from Site C (Table 3), taking advantage of what was learned from unintended consequences and continuous improvement emerging in previous months.
Successes and challenges of using AR to frame an RCT
Change is the main reason for AR. The desire for change arises from a critique of the current situation and the view that there could be something better in its place. This concurs with the motivation for RCT-based research – to identify and examine an intervention that has the potential to improve the health of individuals and/or communities. In answering the case’s research question our aim was to influence national policy on the development of telehealth as a valid, funded, and cost-effective choice for delivering health services that have a likelihood of improving health outcomes for people with chronic conditions. As the patients and their clinicians and service managers learned from their participation, processes and attitudes changed, and improvements occurred, thereby translating research into action. Many of the patients who participated were reluctant to return the telemonitoring equipment as they claimed it had made a difference in their lives. This raised an ethical debate about the consequences of withdrawing a research intervention when a project (and its funding) comes to an end.
We were new to applying AR characteristics and cyclic iterative learning to quantitative studies. At times this pluralism was natural and at other times the interpretivist nature of AR and positivist underpinning of RCTs seemed incommensurate, e.g. potential patient participants were excluded from co-definition of the problem and co-design of the intervention and RCT. For the most part the different approaches worked well together and offered opportunities for more rigour, stronger analysis of the data, and more useful findings. We did experience some challenges. Some of the researchers were strongly positivist, making it difficult for them to see value in what they saw as subjective, biased, and unreliable practices, e.g. positioning researchers inside the research and applying results as they become available. The positivist requirement to generalise findings that are bias free appears to be incommensurable with the interpretivist requirement for transferability and trustworthiness of findings. The main challenge was the lack of funding to build AR deliberately into the overall research project, ensuring that the discipline of practicing and documenting reflexivity was regularly practiced in meetings. The effects of adjusting the intervention during implementation (after learning from the implementation at other sites), and the effects of the research on the organisational context, could have been more rigorously investigated and reported. This could have informed future decisions about including possible patient participants in the early stages of the research. A significant challenge was the increasing degree of distance (and decreasing degree of influence) between the researchers and implementers of the trial, limiting the opportunities to document, analyse, share and apply any influence of AR practices on the trial.
The project leadership changed, resources became scarce for continued participation of academics, and the research/AR team shrank as researchers and collaborators turned to the demands of their daily work lives because the discipline of AR was not documented in the research project’s plan. The opportunities for AR contributions from those who initially negotiated AR as a frame became limited and as funding became more focussed on the RCT cycle, the Action Researchers withdrew.
An RCT was conducted to assess the effect of telemonitoring on health-related quality of life, self-care, hospital use, costs and the experiences of patients, informal carers and health care professionals. AR was negotiated as a frame for the RCT and was conducted by Action Researchers new to using AR to frame an RCT, with mixed levels of commitment from the research project’s team members.
We were able to apply an AR cycle to the process of developing and executing an RCT design, and implement sub-cycles to each step of the overarching cycle (Figs. 1 and 2). Initially we aimed to only use the AR cycle but found ourselves using AR characteristics as well. The characteristics of AR were applied, especially in the early stages of the project. AR offers ways to deliberately and systematically collect data that is not usually collected in RCTs. It is interesting to note in our RCT report that although the RCT itself did not reveal a strong argument to implement tele-monitoring, the qualitative data that was collected revealed that tele-monitoring is a valuable intervention, especially from the patient’s point of view .
Why bother with AR? AR and RCTs complement one another in many ways, as can be seen in Table 1. We know that RCT is a good way of measuring the effect of a telehealth intervention, and we know how important it is to design an RCT well . In the critical approach the researchers are saying that the situation under study should be better in some way, i.e. they critique the status quo, identify a vision for improvement and, using AR cycles and characteristics, improve the situation . This could include an RCT to examine the effectiveness of their actions as a sub-cycle of an overall AR cycle . On the other hand, there is a need to extend the scope of an RCT to include other data about the phenomenon being studied, e.g. to deliberately include the data that falls around an RCT that is often discarded due to concerns about bias and confounding factors . In the interests of quality, the process of defining the scope and purpose of an RCT should be documented for other researchers to use: AR offers a rigorous approach to this documentation and process.
AR appears to be at odds with positivist research [28, 29], and this was a challenge at times during our project. Critical, interpretivist and positivist ways of knowing could be seen as mutually exclusive [28, 30]. They could complement one another, as natural responses to different types of knowing [8, 31]. Positivist research is good for identifying efficacy of clinical interventions where causal links are useful in evidence based care. A broad range of theories about telehealth (including tele-monitoring) are therefore embraced, e.g. socio-technical (e.g. technology acceptance model by Davis), health behaviour, and economics theories . AR thus embraces different ways of knowing to enable multiple aspects of an intervention to be researched simultaneously, while also informing the actual implementation of the intervention in the context of an organisation . Including patient participants in the co-definition and co-design of an RCT was initially seen to compromise the execution of the RCT. Instead of conducting a feasibility study, one could include those participants in the early stage of an RCT design process to strengthen the focus and applicability of the intervention and the RCT itself.
The inclusion of evidence from RCTs into evidence based care is challenging. Clinicians are unable to absorb the rising number of RCT publications, of which the quality is variable . Furthermore, there is a time lag between the performance of an RCT, its publication and the use of its results in everyday clinical practice, further complicated by the slow moving organisational and regulatory environment in which healthcare is practiced [29, 34]. AR offers opportunities to initiate change where change can be implemented as the result of incremental, cyclic improvements. Formative evaluation that is concurrent to an RCT offers opportunities for change in medical practice that are not available in the summative orientation of the RCT process itself .
Formative and summative evaluation of a tele-monitoring intervention, framed by AR, and involving Information and Communications Technology (ICT), implies a form of ‘action design research’ . This approach aims to meet the needs of practitioners by designing an ICT artefact for organisational implementation, while simultaneously conducting theoretical research using formative and summative research methods. ICT is not an intervention in the same way as a medication or surgical procedure is; it is an ‘ensemble artefact’ incorporating clinical and organisational processes and structures into the technology . Consequently, when measuring the tele-monitoring impact on clinical outcomes for people with long term conditions the research data generated represents the embedded processes and structures as well as cause and effect links.
An RCT is not enough to demonstrate whether the ‘intervention’ works or not . It is not enough to design complex RCTs as proposed by Campbell et al. , or to create continuous RCTs to accommodate rapid technological changes as proposed by Mohr et al. . It is not enough to conduct a feasibility study or to simply include qualitative analyses arising from an RCT . We propose that the AR process and characteristics enable telehealth researchers to evaluate formatively and summatively, and enable research teams and collaborators to bring about change as and when opportunities arise without compromising the quality of an RCT.
AR as a frame for planning and executing RCTs offers a structure, processes and characteristics. Researchers with different takes on the research question are able to explore their different ways of knowing, e.g. organisational researchers using social sciences, technology researchers using socio-technical theory and clinicians using positivism to establish causal effects. Action research and positivism both aim to improve a situation. At times AR and RCTs are at odds with one another, but with AR as the overarching frame that allows for democratic planning (and permits different types of participants), responsiveness to emergence at appropriate times, and iterative learning and emergence, then the data that falls around an RCT can, and should, be used purposively.
Our research team agreed to use AR to frame an RCT to assess the effect of tele-monitoring on health-related quality of life, self-care, hospital use, costs and the experiences of patients, informal carers and health care professionals. We were new to the idea of deliberately framing an RCT as an AR project and we learnt how to work together within a communicative space to build an RCT that reflected different ways of knowing. It was more than a mixed methods study. Our study accommodated theoretical pluralism (although challenging at times, especially when the different approaches appeared to be philosophically incommensurable, and when funding limited our capacity to use the discipline associated with AR) to enable researchers representing different ways of knowing. At times doing an RCT framed by AR seemed clumsy, but the way the cycles in Figs. 1 and 2 worked meant that certain ways of knowing dominated depending on the purpose of that cycle.
We propose that the cyclic framework plus the characteristics of AR should be used in the planning, execution and close-off of RCTs to enable stronger design, richer data collection and earlier adoption of evidence into practice for patients, clinicians, the multi-disciplinary team, investors, and improvement of the healthcare environment . AR and RCTs share a purpose – to make improvements. AR enables clinicians and other healthcare practitioners to contribute to research and return the results into everyday practice. Based on our experience, we recommend that all RCTs could be framed as an AR project to enhance the quality, value, and impact of the research on everyday health care.
AR, action research; COPD, chronic obstructive pulmonary disease; ICT, information and communication technologies; PDSA, Plan, Do, Study, Act (cycle); RCT, randomised controlled trials
We wish to thank those involved in the RCT from the different services involved, the university researchers, the project team, and Uniservices, upon which this article is based.
The RCT was funded by the New Zealand Ministry of Health and District Health Boards New Zealand. This report is independent research arising from reflection on the methodology by the three authors and was not funded.
Availability of data and materials
This article is a reflection on the use of Action Research as part of the RCT reported in Kenealy, T. W., Parsons, M. J., Rouse, A. P. B., Doughty, R. N., Sheridan, N. F., Hindmarsh, J. K. H.,…Rea, H. H. (2015). Telecare for Diabetes, CHF or COPD: Effect on Quality of Life, Hospital Use and Costs. A Randomised Controlled Trial and Qualitative Evaluation. PLOS One, 10 . Associated data and materials are available as part of the PLOS One publication.
The idea for the paper was originally developed by KD and TK. TK analysed and reported the RCT used in the case study central to this paper. KD, TK and NS drafted the manuscript, revised the paper critically and approved the final version.
The authors declare that they have no competing interests.
Consent for publication
Consent was granted as part of the consenting process as per the ethics approval.
Ethics approval and consent to participate
Ethics permission for the RCT was granted by the New Zealand Ministry of Health Multi-region Ethics 83 Committee, reference MEC/10/04/031.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. Br Med J. 1996;312(7023):71.View ArticleGoogle Scholar
- Lewis MW, Kelemen ML. Multiparadigm inquiry: Exploring organizational pluralism and paradox. Hum Relat. 2002;55:251.View ArticleGoogle Scholar
- Geertz C. Thick description: Toward an interpretive theory of culture. In M. Martin & L.C. McIntyre (eds.). Readings in the philosophy of social science. Cambridge, Mass: MIT Press; 1994. pp. 213–31.Google Scholar
- Greenhalgh T, Russel J, Ashcroft RE, Parsons W. Why National eHealth Programs Need Dead Philosophers:Wittgensteinian Reflections on Policymakers’ Reluctance to Learn from History. Milbank Q. 2011;89(4):533–63.View ArticlePubMedPubMed CentralGoogle Scholar
- Reason P. Choice and quality in action research practice. J Manag Inq. 2006;15(2):187–203.Google Scholar
- Myers MD. Qualitative research in information systems. MIS Quarterly. 1997;June:241–2.Google Scholar
- Mohr DC, Cheung K, Schueller SM, Brown CH, Duan N. Continuous evaluation of evolving behavioral intervention technologies. Am J Prev Med. 2013;45(4):517–23.View ArticlePubMedGoogle Scholar
- Liu JL, Wyatt JC. The case for randomized controlled trials to assess the impact of clinical information systems. JAMIA. 2011;18:173–80.PubMedPubMed CentralGoogle Scholar
- Kaldoudi E, Chatzopoulou A, Vargemezis V. Adapting the STARE-HI guidelines for the evaluation of home care telehealth applications: An interpretive approach. J Inf Technol Healthc. 2009;7(5):293–303.Google Scholar
- Ekeland AG, Bowes A, Flottrop S. Methodologies for assessing telemedicine: A systematic review of reviews. Int J Med Inform. 2012;81:1–11.View ArticlePubMedGoogle Scholar
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337:a1655. doi:10.1136/bmj.a1655.
- Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ. 2009;339:b3496.View ArticlePubMedPubMed CentralGoogle Scholar
- Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694–6.Google Scholar
- Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8(1):1.View ArticleGoogle Scholar
- Reason P. Integrating action and reflection through co-operative inquiry. Manag Learn. 1999;30(2):207–26.View ArticleGoogle Scholar
- Reason P, Bradbury H. Handbook of action research: Participative inquiry and practice. London: Sage; 2001.Google Scholar
- Dick B. Action research: Action and research. In: Shankar S, Dick B, Passfield R, Swepson P, editors. Effective change management using action learning and action research. Lismore, NSW: Southern Cross Press; 2002.Google Scholar
- Rapoport RN. Three dilemmas in action research with special reference to the Tavistock experience. Human Relations. 1970;23(6):499–513.View ArticleGoogle Scholar
- Kenealy TW, Parsons MJ, Rouse APB, Doughty RN, Sheridan NF, Hindmarsh JKH, et al. Telecare for Diabetes, CHF or COPD: Effect on Quality of Life, Hospital Use and Costs. A Randomised Controlled Trial and Qualitative Evaluation. PloS one. 2015;10(3).Google Scholar
- Baskerville RL, Wood-Harper AT. A critical perspective on action research as a method for information systems research. J Inf Technol. 1996;11(3):235–46.Google Scholar
- Kerr S, Penney L, Moewaka Barnes H, McCreanor T. Kaupapa Maori action research to improve heart disease services in Aotearoa, New Zealand. Ethn Health. 2010;15(1):15–31.View ArticlePubMedGoogle Scholar
- Smith LT. Decolonizing methodologies: Research and indigenous peoples. New York: Zed Books & Dunedin: University of Otago Press; 1999.Google Scholar
- Zhang W, Xu X. Six Sigma and information systems project management: A revised theoretical model. Proj Manag J. 2008;39(3):59–74.View ArticleGoogle Scholar
- Pollock M, Warren J, Gu Y, editors. An approach to active procurement. HINZ 2011 Conference and Exibition 2011; Rotorua, New Zealand. Available from https://www.yumpu.com/en/document/view/37198334/an-approach-to-active-procurement-health-informatics-new-zealand. Accessed on 25 Feb 2016.
- Ash J, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Inform Assoc. 2004;11:104–12.View ArticleGoogle Scholar
- Augestad KM, Bernsten G, Lassen K, Bellika JG, Wootton R, Lindsetmo RO, et al. Standards for reporting randomized controlled trials in medical informatics: A systematic review of CONSORT adherence in RCTs on clinical decision support. J Med Inform Assoc. 2012;19(1):13–21.View ArticleGoogle Scholar
- Baskerville R, Wood-Harper AT. A critical perspective on action research as a method for information systems research. In: Myers MD, Avison D, editors. Qualitative Research in Information Systems. London: SAGE Publications; 2002.Google Scholar
- Susman GI, Evered RD. An assessment of the scientific merits of action research. Adm Sci Q. 1978;582–603.Google Scholar
- Scriven M. A summative evaluation of RCT methodology: And an alternative approach to causal research. J Multidiscip Eval. 2008;5(9):11–24.Google Scholar
- Klein HK, Myers MD. A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Q. 1999;23(1):67–94.View ArticleGoogle Scholar
- Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health. 2003;57(7):527–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Gammon D, Johannessen LK, Sørensen T, Wynn R, Whitten P. An overview and analysis of theories employed in telemedicine studies. Methods Inf Med. 2008;47:260–9.PubMedGoogle Scholar
- Leykum LK, Pugh JA, Lanham HJ, Harmon J, McDaniel Jr RR. Implementation research design: integrating participatory action research into randomized controlled trials. Implement Sci. 2009;4:69.View ArticlePubMedPubMed CentralGoogle Scholar
- Heneghan C, Godlee F. Where next for evidence based healthcare? BMJ. 2013;346.Google Scholar
- Sein MK, Henfridsson O, Purao S, Rossi M, Lindgren R. Action design research. MIS Q. 2011;35(1):37–56.Google Scholar
- Steventon A, Bardsley M, Billings J, Dixon J, Doll H, Hirani S, et al. Effect of telehealth on use of secondary care and mortality: findings from the Whole System Demonstrator cluster randomised trial. BMJ. 2012;344, e3874.View ArticlePubMedPubMed CentralGoogle Scholar
- Orr M, Day K. Knowledge and learning in 'successful' IT projects: a case study. Health Care and Informatics Review Online [Internet]. 2004 (17/06/2004). Available from: http://c.ymcdn.com/sites/www.hinz.org.nz/resource/collection/0F09C2E4-7A05-49FB-8324-709F1AB2AA2F/Knowledge_And_Learning_In_Successful_IT_Projects_Martin_Orr_and_Karen_Day_June_2004.pdf. Accessed 25 Feb 2016.