The complex intervention
Health TAPESTRY (Health Teams Advancing Patient Experience: STRengthening qualitY) is composed of four parts: an interprofessional primary health care team, use of technology to facilitate the collection and sharing of client (patients enrolled in the program) information, trained volunteers to collect information from clients and the facilitation of community engagement and connections [19, 20]. As a whole, Health TAPESTRY aims to enable more coordinated, comprehensive and person-centred care for patients, which are key contributing factors to a strong primary care [21]. Strengthening primary care is an effective way to improve patient and service use outcomes [22,23,24].
Operationally, clients in the program are visited by two community volunteers who ask a comprehensive series of surveys that address the clients’ health needs and personal goals. The information is entered into a tablet application, summarized into a report, which is then received by a small group of interprofessional health care providers (IHPs) (e.g., pharmacist, dietician, physician) linked to the client’s own primary care practice. The group, termed the TAP-Huddle, jointly creates a plan of action based on the report and facilitates the plan. Plans can have a variety of tasks including referrals, follow-up appointments with a physician, or the request for patient friendly resources to be mailed. Data for this study come from the larger trial assessing the feasibility and reproducibility of the results of Health TAPESTRY (see [16] for full protocol).
Study implementation sites and participants
Health TAPESTRY was implemented in Family Health Teams (FHTs) in six communities across Ontario, Canada. FHTs are primary care organizations that formally links physicians and a variety of health care professionals such as nurses, dietitians, and occupational therapists [25]. The interprofessional team members may be co-located or at different locations, depending on the local FHT context. One FHTs had two sites resulting in two TAP-Huddles. Eligible study participants were primary care staff involved with the program either as a TAP-Huddle member (e.g., dietitian) or involved in the management of the program (e.g., administration). Leaders within the primary care practices helped the research team determine the appropriate individuals to invite to participate throughout the duration of the study as the membership of the huddles changed over time.
Data collection
The IHPs were invited to complete the survey through an electronic survey link (i.e., LimeSurvey [26]). The research team invited participants to complete the NoMAD six times over the course of one year during implementation (Fig. 1, [5]). The authors of the NoMAD encourage users to apply any necessary modifications to the instrument to improve its relevance to the study’s objectives [5]. The word intervention in each question was replaced with ‘Health TAPESTRY’ and one question was re-worded in the reflexive monitoring subscale. The original question read; “I am aware of the reports about the effects of [the intervention]” and the question was re-worded to; “I am aware of reports about the effects of Health TAPESTRY". The surveys were voluntary, and participants could withdraw at any point. Informed consent was obtained for all participants. Data was collected from April 2018 to January 2020. This study received ethics clearance from the Hamilton Integrated Research Ethics Board (#3967) and all methods were performed in accordance with relevant guidelines and regulations.
Scoring and feedback summaries
The NoMAD is a 23-item questionnaire; the first three questions are general questions about the intervention, and the remaining 20 questions are more detailed. The NoMAD has two categories of answers for each of the 20 detailed question; Option A had five response options (strongly disagree to strongly agree) and Option B, which participants used if the question was irrelevant to their role, the intervention or to the stage in implementation. While the NoMAD does not have a structured scoring template, we applied a five-point Likert scale to all responses (1 = strongly disagree, 3 = neither agree nor disagree, 5 = strongly agree) used by Gillespie et al. [15]. One item within the collective action subscale was reverse coded. The three general questions at the start of the NoMAD (rated on a scale of 0–10, with higher scores more favourable to normalization) address how familiar the intervention feels (called Familiar), if the intervention feels part of the respondents work currently (called Current), and if the respondent feels the intervention will become a normal part of work (called Future).
To provide the TAP-Huddles feedback regarding the implementation of Health TAPESTRY, we sent aggregate participant data for the site four times over one year to each site’s TAP-Huddle lead, and anyone else at the site who requested it (Fig. 1). The feedback was presented resembling a resembling a traffic light, a method based on Reeve et al. [27]. The traffic light summary (TLS) provided a visual representation of how the TAP-Huddle is understanding and normalizing the program at that time point. The colour reflects the mean (1.0–1.5 = dark red, 1.6–2.5 = red, 2.6–3.5 = yellow, 3.6–4.5 = light green, 4.5–5.0 = dark green). The feedback contained data for each question, and the NPT constructs. A written narrative of the values accompanied the TLS when it was sent to a team. If a team was moving in a negative direction (i.e., Health TAPESTRY was becoming less normalized) compared to the previous time point, that was indicated with both the colour coded visualisation and an extra flag that was indicated with both the colour coded visualisation, extra flag and a written statement in email correspondence (Fig. 2). No further instructions were provided to the TAP-Huddle leaders. Email correspondence and meeting notes were used to gather each TAP-Huddle’s impression of, and feedback on, the NoMAD and TLS as well as document any challenges to implementing the NoMAD.
Data analysis
For insight into the usefulness and key learnings regarding the use of the NoMAD and TLS, we completed an audit of the research team’s emails, meeting minutes and feedback from key individuals involved in the implementation and evaluation of Health TAPESTRY. We conducted several psychometric assessments. We calculated Cronbach’s alpha (α) for each subscale by time point (pooled by site) to assess internal consistency. A value of 0.70 or higher was deemed satisfactory; a score of 0.90 was deemed high [20]. To explore possible floor and ceiling effects of the NoMAD, we calculated the range of scores at each time point and the skewness of each subscale, pooled across site. Skewness values between + 1 and -1 were identified as satisfactory. To assess construct validity, bivariate correlations were calculated between subscales and between subscales and the general questions (Familiar, Current, Future) by time point, pooled across site. Correlations were classified as strong (± 0.70), moderate (± 0.40 to ± 0.60), and weak (± 0.30 to ± 0.10) [21]. To assess concurrent validity, hypothesized NoMAD differences were explored between baseline versus 12-month time points pooled by site (and separately by site). We also examined hypothesized differences between first-time implementers versus second-time implementers at baseline. Given our expected small sample size, no planned statistical analyses were completed to test this hypothesis, rather patterns of difference (using means and standard deviations) were visually inspected and described to inform future work. All analyses were done with IBM SPSS (Version 26) software.