Skip to main content

Table 3 Evaluation design

From: Multiple and mixed methods in formative evaluation: Is more better? Reflections from a South African study

Objectives

Methodological approach used in the sub-study that addressed each objective (study sites in which the sub-study was conducted)

Rationale for the methods chosen

Implementation and data analysis

Objective 1

To describe the programme and its implementation

Sub-study 1: Time-and-motion, in which the LHWs’ activities and time spent on each activity were recorded. (Sites 1–3)

1) To develop a detailed description of LHW activities, including any variation between sites and to gain insight into how LHWs organized their work

2) To understand how the programme was implemented to help better understand the contextual issues impacting on the programme

1) Each LHW was shadowed for one day by an evaluator with a stop-watch and a checklist for recording time per activity

2) Data were categorised into client-contact, walking between houses, and waiting time

3) Descriptive analysis in Excel was performed

Sub-study 3: Structured observations of the content and nature of the interactions and communication between LHWs and clients during the LHW visits. (Site 1)

To obtain in-depth information about how the LHWs provide treatment and adherence support in the field

1) LHWs with whom we had good rapport were asked to identify adherent and non-adherent clients

2) An evaluator used an observation guide to record (i) interactions between LHW and client; (ii) mood during the visit; (iii) content and pattern of the conversation; and (iv) any additional issues that impacted on the visit

3) The visit was audio-recorded

4) The recordings were transcribed and analysed thematically

Objective 2

To solicit the experiences and perceptions about the programme from clients, lay health workers, and programme managers

Sub-study 2: Anonymous client survey (Sites 1–3)

To gain information about clients’ experiences and assessments of the programme

1) The questionnaire was administered in the three main languages of the respective sites

2) Clinic staff distributed the information letters to all TB, HIV/AIDS and co-infected clients as they attended the clinic, and referred those interested in participating to the evaluator, who administered the survey at the clinic

3) Descriptive analysis in Excel was performed

Sub-study 4: Interviews with clients, LHWs, NGO staff and managers within the relevant health authorities (Sites 1–3)

To obtain in-depth data on the views of the participant groups, in order to complement the quantitative data from the time-and-motion and survey sub-studies

1) Interviews and conversations with LHWs during the time-in-motion study, and impromptu conversations with clients during LHW visits were audio-recorded

2) Interviews with NGO staff and health authority managers

3) The recordings were transcribed and analysed thematically

Sub-study 5: Photo and voice diary keeping by clients

(Sites 1, 2)

To gain an in-depth understanding of how clients cope with their illness and the role that LHWs play in this. The diaries were trialed as an alternative to interviews

1) Participants received a disposable camera and/or audio recorder and we asked them to share how they coped with their illness/es, including their everyday interactions with LHWs and clinic staff, and the barriers and enablers to treatment adherence. These images and recordings were collected and discussed with the participant during monthly visits. Conversations were audio-recorded

2) The data were thematically analysed