Skip to main content
Download PDF

Combining the theory of change and realist evaluation approaches to elicit an initial program theory of the MomConnect program in South Africa

BMC Medical Research Methodology volume 20, Article number: 282 (2020) Cite this article

Abstract

Background

One of the Sustainable Development Goals is to reduce the global maternal mortality ratio to less than 70 per 100,000 live births by 2030. In South Africa, the flagship National Department of Health MomConnect program was launched in 2014 to strengthen the quality of maternal and child health (MCH) services and improve mortality outcomes. MomConnect was rapidly rolled out with a limited understanding of how and why the program was expected to work even though studies had shown the effectiveness of the MomConnect program in improving the uptake of MCH services. This study aimed to unearth the initial program theory of the MomConnect program based on explicit and implicit assumptions of how the program was organized and expected to work.

Methods

We conducted a document analysis using design- and implementation-related documents of the MomConnect program guided by the principles of Theory of Change (ToC) and Realist Evaluation (RE). Content and thematic analysis approaches were deductively applied to analyze the documents toward constructing ToC and RE-informed models. Abductive thinking and retroduction were further applied to the realist-informed approach to link program context, mechanisms, and outcomes to construct the initial program theory.

Results

ToC and RE-informed models illustrated how the MomConnect program was organized and expected to work. The process of constructing the ToC provided the platform for the development of the initial program theory, which identified three critical elements: (1) the central modalities of the MomConnect program; (2) the intended outcomes; and (3) the tentative causal links indicating, in a stepwise manner of, how the outcomes were intended to be achieved. The RE approach ‘enhanced’ the causal links by identifying relevant programmatic contexts and linking the postulated mechanisms of action (empowerment, encouragement, motivation, and knowledge acquisition) to program outcomes.

Conclusion

The application of ToC and RE provided an explicitly cumulative approach to knowledge generation in unveiling the initial program theory of MomConnect rather than delivering answers to questions of program effectiveness.

Peer Review reports

Background

Target 3.1 of the Sustainable Development Goals (SDGs) aims to reduce the global maternal mortality ratio (MMR), defined as the number of maternal deaths attributed to pregnancy-related complications per 100,000 live births, to less than 70 deaths per 100,000 live births by the year 2030 [1]. Achieving this ambitious goal will require a concerted effort from respective country health authorities due to the numerous challenges impeding the prevention of maternal mortality [2].

With an institutional MMR of 133.3% per 100,000 live births in 2013/2014 [3] and 105.9 in 2018/2019 [4], South Africa (SA) has made notable progress in reducing its MMR. Although SA failed to achieve the Millennium Development Goals (MDG) 4a and 5a of reducing under-five mortality by two-thirds and maternal mortality by three-quarters between 1990 and 2015, significant progress was observed [5]. However, this figure is still high and far from the SDG target. The high MMR in SA is in part attributed to the lack of adequate maternal health care complemented by underlying social determinants of poor nutrition and poverty [6, 7].

Although SA’s MMR has improved from 2014 to 2019, complications of hypertension in pregnancy and obstetric hemorrhaging are still the main issues causing maternal mortality, which can be prevented through early uptake of antenatal care (ANC) and postnatal care (PNC) services [8]. The first ANC visit allows for easy detection of pregnant women requiring special attention, referral, and more ANC visits. Early ANC visits are also associated with adherence to antiretroviral therapy during pregnancy among HIV positive women, which reduces the chances of mother to child transmission [9]. Data from SA show that the rate of ANC visits within 20 weeks of gestation was 44.0% in 2012/2013 [9], 53.8% in 2014, and 68.1% 2018/2019 with a different point percent of 14.3 since 2014 [4]. PNC visits within the first 6 days of delivery’ were 73.0% in (2013/2014) [10].

In keeping with the SDG 3 target, SA developed a strategic plan to reduce maternal and child mortality by improving the uptake of maternal and child (MCH) services. To achieve this goal, mobile health technology (mHealth) was integrated into the healthcare system as a strategy to overcome barriers to universal health coverage and promote MCH [11]. The National Department of Health (NDoH) in 2014 launched the flagship MomConnect program to strengthen the quality of MCH services and improve mortality outcomes [12]. The MomConnect program was specifically adopted to address the peripartum factors perpetuating the high maternal and infant mortality rates in SA [12].

The MomConnect program is a digital health communication program [13], which was informed by lessons, experiences, and successes of the MAMA SA program [14, 15]. MAMA SA focused on the prevention of mother-to-child transmission (PMTCT) of HIV and consisted of a multi-channel mHealth approach to communicate healthy pregnancy and newborn child support including mobile phone-based chats and voice messages [16].

The MomConnect program was also guided by an emergent body of work around the use of mobile phones to enhance MCH programs [16,17,18,19,20,21]. Evidence suggests that communicating with pregnant women and new mothers via regular SMS and/or voice-based messages supports the adoption of healthy behaviors and increases the uptake of health services [12]. The MomConnect program leveraged elements from the MAMA SA program based on research to determine the content that would be provided in combination with the NDoH’s messaging on MCH and MCH services.

The program aimed at (i) registering all pregnant women using the public health sector; (ii) sending targeted health promotion messages during pregnancy and up until the first year of age to encourage healthy and health-seeking behavior [14]; and (iii) providing pregnant women with a feedback platform to rate the services they received [18]. MomConnect is a two-way communication channel, providing women the opportunity to rate the MCH services received at the clinics and ask questions regarding their health and that of their babies.

Although no study has shown the effectiveness of the MomConnect program in improving the uptake of MCH services, there is evidence that the national ANC visits within 20 weeks of gestation rate have been increasing in the 3 years following the implementation of MomConnect [4, 10]. Even though MomConnect has a framework for explaining how the program is expected to contribute to expected outcomes, the mechanisms underlying and health systems context that influence the outcome of the MomConnect program remain elusive. Despite this limited understanding, MomConnect was rapidly rolled out. An important step in improving the impact of MomConnect is by unearthing its program theory – the explicit and implicit assumptions of how the program is expected to work.

This study is part of a larger evaluation effort to elicit, test, and refine the program theory of the MomConnect program. The larger study has three phases: Phase 1 will focus on eliciting the initial program theory using three different approaches: a scoping review, document review, and stakeholders’ interviews. Phase 2 will be based on testing the initial program theory, while Phase 3 will entail refining the initial program theory [22]. This paper is part of Phase 1 whereby we sought to glean the initial program theory of MomConnect based on analysis of program-related documents using a combined Theory of Change (ToC) and Realist Evaluation (RE) approaches.

Methodological approaches

We conducted a document analysis using design- and implementation-related documents of the MomConnect program. Document analysis is a systematic procedure for reviewing or evaluating both printed and electronic (computer-based and Internet-transmitted) material [23]. Our document analysis was informed by the ToC and RE approaches [24].

ToC and RE are members of the theory-driven evaluation (TDE) or theory-based evaluation (TBE) family. TDE/TBE are regarded as key to untangling the multiple processes between policy intent and outcomes by investigating program implementation, and the causal link processes that trigger outcomes [25, 26]. Traditionally, ToC and RE are applied separately in different studies to obtain corresponding outputs. However, we considered the two approaches together in our study to obtain a more robust initial program theory for the MomConnect program.

According to Taplin et al. [27], ToC makes underlying assumptions explicit by delineating the different components: input, outputs, outcomes, and impact [28, 29]. ToC can be viewed as both a product and process [27]. As a product, its inquiry results in specific outcomes in a narrative and/or visual form and offers a framework for sense-making that needs to be used, revisited, and adapted as the project or program progresses. As a process, the ToC model shows how a goal will be reached [27].

While ToC illustrates ‘how’ change occurs through program implementation, RE goes deeper by establishing ‘why’ through a mechanism-based generative causal framework. Blamey and Mackenzie [30] propose that the ToC can be used as a means of explicating intervention implementation theory for program planning, improvement, and the development of robust monitoring systems at a whole program level; while RE can be used to examine, in detail, how and why the different aspects of the program leads to the observed or intended outcome – program theory (Fig. 1).

Fig. 1
figure 1

Relationship between implementation theory and program theory. Adapted from Eastwood et al., [31]

Full size image

RE focuses on providing a ‘causation-based explanation’ – depiction of cause-effect relationships among elements [24] – to provide explanations of program effectiveness [32]. RE is a theory-driven approach to evaluation informed by principles of critical realism – social structures and agents having underlying causal powers that interact to cause an observed behavior [33]. RE aims to unearth theories to explain how and why the intervention is supposed to trigger change [34]. RE theories are formulated by conceptualizing the relationship between the context (C) within which the program is implemented, the generative mechanisms of change (M), and the observed outcomes (O). This conceptualization is achieved by formulating the Context-Mechanisms-Outcome configurations (CMOs) [35]. Of importance, nevertheless, is the notion that the Intervention (I) must be accepted by the users, Actors (A), for it to be successful. Therefore, we considered adding elements of the Actors (A) and intervention (I) modalities to the original CMOs heuristic tool and adopted the intervention-context-actor-mechanism-outcomes (ICAMO) configuration [33].

There are multiple definitions of ‘mechanism’. In RE terms, a mechanism (M) relates to a combination of intended and unintended resources provided by the program and the response to those resources by stakeholders [36]. A mechanism is also defined as the reasoning and responses that participants attribute to the resources, opportunities, and constraints of a program to bring about the observed outcomes [37, 38]. Others have defined mechanisms as the underlying entities, process, or structures, which operate in a particular context to generate outcomes of interest [38]. Context (C) is the physical or perceived conditions, which allow the mechanisms to come into operation or remain inactive. The context can also be defined as circumstances that configure the settings in which the intervention takes place, and the action sets of activities that will trigger the change [24]. An outcome (O), which can be represented in short and intermediate terms, is what is observed resulting from the interaction between the context and mechanism that is triggered by the intervention, and could be measured as the impact of the intervention [39].

By combining the ToC and RE approaches we intended to unearth, in a systematic manner, the assumptions underlying the MomConnect program and speculate the changes likely to take place as a result of the intervention [40]. Our intention to glean the initial program theory of the MomConnect program informed by ToC and RE approaches was based on harnessing the overlapping and complementary features of these two approaches. To this end, we adopted a framework proposed by Dhillon and Vaca [24] (Fig. 2) integrating the features of both ToC and RE approaches and how they complement each other to inform the elicitation of a robust and testable program theory.

Fig. 2
figure 2

A framework illustrating the features of ToC and RE (Dhillon and Vaca [24])

Full size image

Methods

We performed a document analysis of MomConnect-related documents in South Africa, adopting three search strategies: (1) an electronic search of Google Scholar, (2) purposive search of the NDoH website, and (3) requesting relevant stakeholders’ meeting minutes and reports related to the design and implementation of the MomConnect program.

The Google Scholar search covered the period 2014–2019, i.e. from the year that MomConnect was implemented until the year the document review was initiated. Using the terms ‘MomConnect intervention’ OR ‘MomConnect program’ resulted in 266 hits. Combining the terms (‘MAMA South Africa’ AND ‘MomConnect program’) resulted in 86 additional documents, resulting in 352 documents. Eighteen documents relating to the MomConnect program were found by searching the NDoH website. Ten meeting minutes and reports were retrieved from key stakeholders resulting in a total of 380 documents (Table 1).

Table 1 Document sources and numbers obtained
Full size table

After screening by title and abstract, 358 documents and duplicates were removed. Screening of the documents for eligibility hence identified 22 documents, which were included in the final analysis (Fig. 3). The characteristics of the selected documents are summarized in Additional file 1.

Fig. 3
figure 3

Document selection flowchart

Full size image

Data analysis

Our data analysis involved skimming (superficial examination), reading (thorough examination), and identifying themes within the documents associated with the elements of ToC and the concepts of RE theory formulation. This iterative process combined elements of content and thematic analyses [23]. Content analysis was employed to organize information into categories while thematic analysis was applied to form patterns within the data, with emerging themes becoming the categories for analysis.

Inductive coding and deductive categorizing were performed to uncover and align themes related to the ToC framework (Fig. 1). After obtaining the ToC, further data coding was applied to identify statements or portions of the texts attributable to the constructs of mechanism and contexts. Abduction – inventive thinking required to imagine the existence of mechanisms and retroductive thinking – a mechanism-focused approach to inference making, which seeks to clarify the basic prerequisite or conditions of a phenomenon, were applied to explore and link elements of intervention modalities, context, actors, mechanisms, and outcomes to obtain the ICAMO configuration.

Results

Figure 4 presents a summary of the themes obtained from the content and thematic analyses. The impact was identified as reductions in maternal, neonatal, and child morbidity and mortality. Outcomes were related to health-seeking behavior; improved MCH services uptake, and improved quality of MCH services. The output themes speak to interactions with the health system and knowledge acquisition.

Fig. 4
figure 4

Identified themes based on Theory of Change framework

Full size image

MomConnect program theory (ToC)

The process of constructing the ToC provided three critical features toward developing an initial program theory of the MomConnect program. Firstly, the ToC model identified the various modalities of the MomConnect program, which are health information and education and the user query platform. Secondly, it clearly articulated the intended outcomes (short-term and long-term) of the MomConnect program. Finally, the ToC provided tentative causal links in a stepwise manner to show how the outcome is to be achieved (Fig. 5).

Fig. 5
figure 5

The MomConnect program theory of change

Full size image

The realist evaluation-informed model

Our ToC underpinned the process of developing the initial program theory, which is based on the RE methodological approach. As highlighted in Fig. 1, additional elements are required to make the shift from a ToC to a realist program theory. Notably, the identification of mechanisms and contextual elements are critical to establishing a generative causality – how the intervention modalities are interpreted and acted upon by the actors (mechanisms) and under what conditions (context) leading to the hypothesized impact. Table 2 illustrates a thematic analysis of the data based on RE principles and the ICAMO heuristic tool.

Table 2 Intervention, context, actors, mechanism and outcome elements of realist evaluation
Full size table

Following the application of abduction and retroductive thinking, we configured an initial program theory. Table 2 presents the thematically obtained elements used to develop the RE model (Fig. 6), which links the Intervention (I), context (C), causal pathways of outcomes (O), and the mechanisms (M) describing the causal chains, or causal pathways that explain how, when and/or why the MomConnect program messages trigger the expected results.

Fig. 6
figure 6

Realist evaluation model for MomConnect program

Full size image

A key component of the program consists of sending pregnancy-related stage-based messages (I) to pregnant women and mothers (A), facilitated by health systems conditions such as facility readiness to provide care (C), and intervention-related resources and mobile connectivity (C). The pregnancy-related stage-based information was envisaged to encourage and empower (M) users through knowledge acquisition to motivate them (M) thus improving their health-seeking behavior (O).

Interaction with the health system (service rating and feedback) (I) is influenced by health system conditions such as facility readiness to provide care (C) and health care worker availability and buy-in (C). By rating the services received at the facility level, women and mothers are expected to feel empowered (M), which in turn would result in improved quality of health care delivery (O) and improved uptake of MCH services (O).

Discussion

We aimed to glean an initial program theory of the MomConnect initiative to understand how and why the program works (or not). We used the ToC and RE approaches to elicit the initial program theory [41]. Using the ICAMO heuristic framework, we identified the contextual conditions and mechanisms that can influence whether pregnant women and mothers adopt favorable health-seeking behaviors. Our study identified three main contextual conditions – facility responsiveness to provide care including human resource availability, health system context such as political clout, and mobile connectivity related context – and four causal mechanisms – empowerment and perceived quality of MCH services, encouragements, motivation, and knowledge acquisition.

Findings from the study revealed that if pregnant women and mothers receive pregnancy related information by ‘SMSs’, they will become sensitized about existing MCH services tailored to the stage of their pregnancy. Being sensitized can encourage and motivate them to attend clinic appointments, ultimately decreasing maternal and child mortality. We also found that the interaction with the health system (service rating and feedback) provided by the MomConnect program is likely to empower users, since they feel that it can improve the quality of services offered at healthcare facilities, thus improving the perceived quality of MCH services and, in turn, the uptake of health services.

These findings are consistent with the work by Ruton et al. [42] in which CHWs were given mobile phones to report data on MCH indicators using text messages (Rapid SMS a two-way communication). The reminders for clinic appointments on ANC, the probable delivery date, and PNC were generated automatically. These reminders were sent to the CHWs who in turn contact the clients, resulting in an increase in routine care attendance and facility deliveries. RapidSMS influenced the use of MCH services, whereby an additional support package was provided such as training of health care providers (HCPs), equipment, and supervision [42]. Our finding is also consistent with the mHealth ImTeCHO intervention in India [43], which established a significant relationship between the use of mobile phone technology by HCPs and changes in the working environment. Training, supervision, workload, and remuneration motivated HCPs to perform well in their duties.

Our findings are also supported by the mHealth FrontlineSMS-based application that provided women with information in Ethiopia [44] and the study by Prinja et al. in India [45], which used a ToC approach to explain how the use of mHealth supported and improved the quality of counseling delivered by HCPs. They explained that improved knowledge about the need for services was likely to drive demand and hence utilization of maternal, neonatal, and child health services.

A study conducted by Hategeka et al. [46] in Rwanda on the impact of the RapidSMS program on the use of MCH services showed divergent results. The lack of impact of RapidSMS on ANC services uptake was attributed to the infidelity in the implementation of the program. The study conducted in Rwanda by Musabyimana et al. [47] on implementers’ and community health workers’ experiences of the RapidSMS program also identified challenges to the successful implementation of the program, which included lack of motivation and incentives, high turnover rates of CHWs, and lack of regular CHW training. These findings are indicative of important contextual elements that can influence the success of the MomConnect program.

Our initial program theory is consistent with some existing models such as the Elaboration likelihood model (ELM) and the extensive Elaboration likelihood model (eELM) [48], and the Theoretical Domains Framework (TDF) [49] in behavior change. ELM and eELM by Adam et al. [48] focused on Mobile Video intervention, which stipulates that the changes in attitude that predict a desired behavioral outcome such as exclusive breastfeeding can be explained in two pathways. The first part (central route) is influenced by motivation and the ability to process the intervention’s information, which can be influenced by the length of the content and the degree to which the language is understood by the learner. The second part (peripheral route) relies on clues embedded in the information delivery method, which contributes to its relative acceptability by the user. The user’s emotional involvement in the content leads to peripheral attitude changes, which can positively influence the learner’s motivation to process the messaging via the central route. Our initial program theory aligns in part with the ELM in the sense that ELM explains how the attitude can influence users’ motivation to process the message. The empowerment and encouragement derived from MomConnect’s messages allow clients to become motivated toward perinatal health care, which in turn improves the uptake of MCH services.

The Theoretical Domains Framework (TDF) was developed to assess implementation problems, support intervention design, and investigate the fidelity of intervention delivery [49]. TDF posits that a good understanding of psychological capability and reflective motivational processes in knowledge, skills, social/professional role and identity, beliefs about capabilities, optimism, beliefs about consequences, reinforcement, intentions, goals and memory, attention and decision processes can influence behavior. TDF explains that the social influences and environmental context and resource domains point to organizational and systems context for change [49]. The TDF supports our model in the sense that, some of the TDF domains were included in our model, such as knowledge acquisition, which is a central modality provided by MomConnect messages through health education and information (I). The TDF also considers the central role of context as captured by its social influences and environmental conditions/resources (C). The TDF also considers the notion of the ‘mechanism of action’ in the domain of beliefs about capabilities and emotion (M). Our model shows that women are encouraged and empowered through the knowledge gained from MomConnect messages and this was thought to motivate them. Our model differs from the TDF in that the TDF offers little conceptual configurations (causal links) between the elements in their different domains.

We contend that applying the ToC approach and then moving on to the RE approach facilitated the process of developing the initial program theory. The ToC was used to explicate the implementation theory for program planning, improvement, and the development of robust monitoring systems at a macro program level [30]. The RE approach was then used on a more micro-level (user-oriented understanding) of how and why the intervention would work or not. The benefit of combining the ToC and RE is shown by the fact that ToC makes it easier to trace causes if outcomes are not achieved at the management level. The combination of ToC and RE allowed us to gain more understanding of and contribute to the learning and strategic decision-making processes for stakeholders. Also, the merging of these two approaches promote a better understanding of the successes and challenges of the MomConnect program. Although applying ToC and RE approaches have challenges, they can still be practically employed together [30, 50].

Rigor and trustworthiness

The formulation of the ToC and initial program theory was achieved in an iterative process – moving back and forth between the data, the ToC, and developing the initial program theory. All authors held discursive weekly meetings to conceptualize and finalize the ToC and initial program theory.

Strengths and limitation

Overall, our study has established theoretical foundations and comprehensive requirements for the development of program theory in mHealth interventions (MomConnect), to support decision making, and decision support capabilities. Our study contributes to the cumulative theory development activities, mHealth, MCH services, and decision support. The program theory and connection between mHealth, improved MCH services, and decision making is highlighted in our ToC and RE, and the implications for developers of mobile-based interventions are that the model provides a holistic approach to system thinking, systems analysis, and design. This assisted in actualizing the next phases of our study, which is the development of a program theory using a scoping review and stakeholders’ interviews.

The study has several limitations as both ToC and RE have their challenges. For instance, the ToC model tends to be linear and lacking detail, particularly around causality. The model can be too descriptive and ignore issues of power and conflicting theories. ToC is also time and resource-intensive and has some difficulties in dealing with multiple perspectives, unexpected outcomes, and critical analysis. In RE, on the other hand, it can be difficult to identify and conceptualize mechanisms and separate context from mechanisms. RE lacks substantive theory in some fields, multiple and conflicting theories in others, and uses inaccessible jargon. Like ToC, RE is also time and resource-intensive. A final limitation is that this study only included document reviews and excluded studies such as RCTs and systematic reviews.

Conclusion

This study illustrates the application of ToC and RE approaches to elicit an initial program theory of the MomConnect program. Because both approaches take an explicitly cumulative approach to knowledge generation, they have been usually applied in isolation. Based on the experience from our study, we concur that ToC and RE approaches can be practically employed together and that their synergism can partially overcome the critiques of each methodology. We found that the application of both ToC and RE enhances learning and the unveiling of the initial program theory rather than delivering answers to questions of program effectiveness.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files - Additional file 1.

Abbreviations

ANC:

Antenatal care

ART:

Antiretroviral therapy

CHWs:

Community Health Workers

CMOc:

Context Mechanism Outcomes configuration

ELM:

Elaboration likelihood model

eElm:

Extensive Elaboration likelihood model

HCP:

Health Care Providers

ICAMO:

Intervention Context Actors Mechanism Outcomes configuration

MCH:

Maternal and Child Health

MMR:

Maternal Mortality Ratio

MNCH:

Maternal, Neonatal, and Child Health

NDoH:

National Department of Health

PMTCT:

Prevention of mother-to-child transmission of HIV

PNC:

Post Natal Care

PT:

Program theory

RCTs:

Randomized Controlled Trials

RE:

Realist Evaluation

SA:

South Africa

SDGs:

Sustainable Development Goals

TBE:

Theory-based evaluation

TDE:

Theory-driven evaluation

TDF:

Theoretical Domains Framework

ToC:

Theory of Change

USSD:

Unstructured Supplementary Service Data

WHO:

World Health Organization

References

  1. Callister LC, Edwards JE. Sustainable development goals and the ongoing process of reducing maternal mortality. J Obstet Gynecol Neonatal Nurs. 2017;46:e56–64.

    PubMed  Article  Google Scholar 

  2. WHO U, UNFPA WBG, the United Nations Population Division. Maternal Mortality 2000 to 2017. 2019, https://www.who.int/news-room/fact-sheets/detail/maternal-mortality.

    Google Scholar 

  3. Massyn N, Day C, Peer N, et al. District Health Barometer DHB 2013/2014: Health System Trust; 2015. https://www.hst.org.za/publications/District%20Health%20Barometers/DHB_2013_14_web.pdf.

  4. Massyn N, et al. E. District Health Barometer 2018/2019: Health System Trust; 2020. p. 1–462. https://www.hst.org.za/publications/Pages/DISTRICT-HEALTH-BAROMETER-201819.aspx.

  5. Damian DJ, Njau B, Lisasi E, et al. Trends in maternal and neonatal mortality in South Africa: a systematic review. Syst Rev. 2019;8:1–13.

    Article  Google Scholar 

  6. Mmusi-Phetoe RMM. Social factors determining maternal and neonatal mortality in South Africa: a qualitative study. Curationis. 2016;39:1571.

    PubMed  PubMed Central  Article  Google Scholar 

  7. Udjo EO, Lalthapersad-Pillay P. Estimating maternal mortality and causes in South Africa: national and provincial levels. Midwifery. 2014;30:512–8.

    PubMed  Article  Google Scholar 

  8. Moodley J, Cheb MB, Fawcus S, et al. Improvements in maternal mortality in South Africa. SAML Res. 2018;108:4–8.

    Google Scholar 

  9. Massyn N. Maternal, Child & Women’S Health Barometer, 2016: Health System Trust; 2018. p. 116. https://www.hst.org.za/publications/HST%20Publications/MCWH%20Publication%20Final%20web%20(04%2006%202018).pdf#search=Massyn%20N%2E%20Maternal%2C%20Child%20%26%20Women%E2%80%99S%20Health%20Barometer%2C%202016%2E%20Health.

  10. Massyn N, Peer N, Padarath A, et al. District Health Barometer 2015/2016: Health system Trust; 2017. p. 1–626. https://www.hst.org.za/publications/District%20Health%20Barometers/District%20Health%20Barometer%202015_16.pdf.

  11. Le Blanc D. Towards integration at last? The sustainable development goals as a network of targets. Sustain Dev. 2015;23:176–87.

    Article  Google Scholar 

  12. National Department of Health South Africa. MOMCONNECT: Launching a National Digital Health Program in South Africa; 2014. p. 28.

    Google Scholar 

  13. Barron P, Peter J, LeFevre AE, et al. Mobile health messaging service and helpdesk for south African mothers (MomConnect): history, successes, and challenges. BMJ Glob Health. 2018;3:e000559.

    PubMed  PubMed Central  Article  Google Scholar 

  14. Seebregts C, Barron P, Tanna G, et al. MomConnect: an exemplar implementation of the Health normative standards framework in South Africa. S Afr Health Rev. Jan 2016;2016:125–35.

    Google Scholar 

  15. Mehl GL, Tamrat T, Bhardwaj S, et al. Digital health vision : could MomConnect provide a pragmatic starting point for achieving universal health coverage in South Africa and elsewhere ? BMJ Glob Health. 2018;3:1–5.

    Article  Google Scholar 

  16. Coleman J. Monitoring MAMA: Gauging the impact of MAMA South Africa. J Mob Technol Med. 2013;2:9.

    Article  Google Scholar 

  17. Atik ARA. Understanding mHealth Impact among Aponjon (MAMA Bangladesh) Subscribers through a Phone Survey in Bangladesh. 2013, https://dl.acm.org/doi/pdf/10.1145/2517899.2517920.

    Google Scholar 

  18. Barnett, I., Brockerhoff, S., Medardi, D., Gordon, J. and Srivastava S. Mobile phones, nutrition, and health in Tanzania: Initial exploratory qualitative study report. 2017, https://opendocs.ids.ac.uk/opendocs/bitstream/handle/20.500.12413/13566/mNutrition_Tanzania_final_23_Aug_2017.pdf?sequence=821&isAllowed=y.

    Google Scholar 

  19. Dalious SCWAR and M. Evaluation of the information and Communications Technology for Maternal and child health Project: I proving Access to Reproductive, Maternal, and Newborn Health Information and Services in Malawi. 2013, http://www.villagereach.org/wp-content/uploads/2017/07/ICT_for_MNCH_Report_131211md_FINAL.pdf.

    Google Scholar 

  20. Evans WD, Abroms LC, Poropatich R, et al. Mobile Health evaluation methods: the Text4baby case study. J Health Commun. 2012;17:22–9.

    PubMed  Article  Google Scholar 

  21. Peter JE, Barron P, Pillay Y. Using mobile technology to improve maternal, child, and youth health and treatment of HIV patients. S Afr Med J. 2016;106:3.

    Article  Google Scholar 

  22. Kabongo EM, Mukumbang FC, Delobelle P, et al. Understanding the influence of the MomConnect programme on antenatal and postnatal care service utilisation in two south African provinces: a realist evaluation protocol. BMJ Open. 2019;9:1–9.

    Article  Google Scholar 

  23. Bowen GA. Document analysis as a qualitative research method. Qual Res J. 2009;9:27–40.

    Article  Google Scholar 

  24. Dhillon L, Vaca S. Refining theories of change. J Multidiscip Eval. 2018;14:64–87.

    Google Scholar 

  25. Chen H-T, Rossi H. Issues in the theory-driven. Eval Progr Planning. 1989;12:299–306.

    Article  Google Scholar 

  26. Stame N. Theory-based evaluation and types of complexity. Evaluation. 2004;10:58–76.

    Article  Google Scholar 

  27. Taplin DH, Clark H, Collins E, et al. Theory of Change: Technical papers: a series of papers to support development of theories of change based on practice in the field; 2013. p. 23.

    Google Scholar 

  28. Mayne J. Useful theory of change models. Can J Program Eval. 2015;30:119–42.

    Article  Google Scholar 

  29. Derbyshire J. Use of scenario planning as a theory-driven evaluation tool. Futur Foresight Sci. 2019;1:e1.

    Article  Google Scholar 

  30. Blamey A, Mackenzie M. Theories of change and realistic evaluation: peas in a pod or apples and oranges? Evaluation. 2007;13:439–55.

    Article  Google Scholar 

  31. Eastwood JG, Woolfenden S, Miller E, et al. Implementation, mechanisms of effect, and context of an integrated care intervention for vulnerable families in Central Sydney Australia: a research and evaluation protocol. Int J Integr Care. 2019;19:1–13.

    Google Scholar 

  32. Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18:176–91.

    Article  Google Scholar 

  33. Mukumbang FC, Marchal B, Van Belle S, et al. Using the realist interview approach to maintain theoretical awareness in realist studies. Qual Res. 2019. https://doi.org/10.1177/1468794119881985 Epub ahead of print.

  34. Masterson-Algar P, Burton CR, Rycroft-Malone J, et al. Towards a programme theory for fidelity in the evaluation of complex interventions. J Eval Clin Pract. 2014;20:445–52.

    Article  PubMed  Google Scholar 

  35. Goicolea I, Vives-Cases C, Hurtig AK, et al. Mechanisms that trigger a good health-care response to intimate partner violence in Spain. Combining realist evaluation and qualitative comparative analysis approaches. PLoS One. 2015;10:1–17.

    Google Scholar 

  36. Pawson R, Tilley N. Realistic evaluation: Sage; 1997.36. Pawson R, Tilley N. Realistic evaluation. Sage, 1997. https://uk.sagepub.com/en-gb/afr/realistic-evaluation/book205276.

  37. Mukumbang FC, Van Belle S, Marchal B, et al. Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol. BMJ Open. 2016;6:1–11.

    Article  Google Scholar 

  38. Flynn R, Rotter T, Hartfield D, et al. A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on the sustainability of a lean intervention in pediatric healthcare. BMC Health Serv Res. 2019;19:1–12.

    Article  Google Scholar 

  39. Pawson R, Tilley N. Realistic evaluation: Sage Publ; 2004. p. 1–36. Pawson R, Tilley N. Realistic evaluation. Sage Publ 2004; 1–36. https://www.communitymatters.com.au/RE_chapter.pdf.

  40. Garret R. Theory of change thinking in practice. Dis Colon Rectum. 1988;31:154–5.

    Article  Google Scholar 

  41. Mukumbang FC, Marchal B, Van Belle S, et al. A realist approach to eliciting the initial programme theory of the antiretroviral treatment adherence club intervention in the Western Cape Province , South Africa. BMC Med Res Methodol. 2018;18:47.

    PubMed  PubMed Central  Article  Google Scholar 

  42. Ruton H, Musabyimana A, Gaju E, et al. The impact of an mHealth monitoring system on health care utilization by mothers and children: an evaluation using routine health information in Rwanda. Health Policy Plan. 2018;33:920–7.

    PubMed  PubMed Central  Article  Google Scholar 

  43. Modi D, Gopalan R, Shah S, et al. Development and formative evaluation of an innovative mHealth intervention for improving coverage of community-based maternal, newborn, and child health services in rural areas of India. Glob Health Action. 2015;8. https://doi.org/10.3402/gha.v8.26769 Epub ahead of print.

  44. Atnafu A, Otto K, Herbst CH. The role of mHealth intervention on maternal and child health service delivery: findings from a randomized controlled field trial in rural Ethiopia. mHealth. 2017;3:39.

    PubMed  PubMed Central  Article  Google Scholar 

  45. Prinja S, Nimesh R, Gupta A, et al. Impact of m-health application used by community health volunteers on improving utilisation of maternal, new-born and child health care services in a rural area of Uttar Pradesh, India. Trop Med Int Health. 2017;22:895–907.

    PubMed  Article  Google Scholar 

  46. Hategeka C, Ruton H, Law MR. Effect of a community health worker mHealth monitoring system on uptake of maternal and newborn health services in Rwanda. Glob Health Res Policy. 2019;4:1–11.

    PubMed  PubMed Central  Article  Google Scholar 

  47. Musabyimana A, Ruton H, Gaju E, et al. Tugas Translate Jurnal Assessing the Perspectives of Users and Beneficiaries of a Community Health Worker Mhealth Tracking System for Mothers and Children in Rwanda Pekerja Kesehatan Masyarakat Sistem Pelacakan Mhealth Untuk Ibu Dan Anak-Anak Di Rwanda Pe. PLoS One. 2018;10:1–13.

  48. Adam M, Tomlinson M, Le Roux I, et al. The Philani MOVIE study: a cluster-randomized controlled trial of a mobile video entertainment-education intervention to promote exclusive breastfeeding in South Africa. BMC Health Serv Res. 2019;19:1–14.

    Article  Google Scholar 

  49. Atkins L, Francis J, Islam R, et al. A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12:1–18.

    Article  Google Scholar 

  50. Rolfe AS. Combining theories of change and realist evaluation in practice : lessons from a research on evaluation study. Evaluation. 2019;25:294–316 Sage Publ.

    Article  Google Scholar 

Download references

Acknowledgments

Not applicable.

Funding

The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Division of Health Systems and Public Health, Stellenbosch University, Cape Town, South Africa

    Eveline M Kabongo & Edward Nicol

  2. South African Medical Research Council, Cape Town, South Africa

    Ferdinand C. Mukumbang & Edward Nicol

  3. School of Public Health, University of the Western Cape, Cape Town, South Africa

    Peter Delobelle

  4. Department of Public Health, Vrije Universiteit Brussel, Brussels, Belgium

    Peter Delobelle

Authors
  1. Eveline M Kabongo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ferdinand C. Mukumbang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peter Delobelle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Edward Nicol
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The draft manuscript was written by EMK. FCM, PD, and EN assisted with comments that critically contributed to developing and refining the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Eveline M Kabongo.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

None declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

List of documents containing clauses for MomConnect program (n = 22) the Document was ordered by date of publication.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

M Kabongo, E., Mukumbang, F.C., Delobelle, P. et al. Combining the theory of change and realist evaluation approaches to elicit an initial program theory of the MomConnect program in South Africa. BMC Med Res Methodol 20, 282 (2020). https://doi.org/10.1186/s12874-020-01164-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12874-020-01164-y

Keywords

  • mHealth
  • MomConnect
  • Theory of change
  • Realist evaluation
  • Maternal and child health

BMC Medical Research Methodology

ISSN: 1471-2288

Contact us