Table 7 Suggestions for optimising the benefits of multiple, mixed-method formative evaluations

• Multiple, mixed method formative evaluations require careful planning to select appropriate methods, develop appropriate data collection instruments, sequence data collection, collect data and undertake analysis in ways that both does justice to the individual methods and allows data to be triangulated across methods

• The evaluation protocol should include information on the methods and approaches that will be used to triangulate, and in some cases integrate, the findings from each of the evaluation methods used

• Consultation with and involvement of key stakeholders, including those commissioning the evaluation, can help to ensure that appropriate methods are selected to address the evaluation questions

• The evaluation plan should include opportunities for the evaluation team to reflect on whether the methods selected are achieving their objectives and whether changes need to be made to the mix of methods selected or their sequencing within the overall evaluation

• Multiple methods can easily overstretch the resources of the evaluation team. A judicious balance needs to be struck between what is practically feasible, in terms of resources, time and the skills of the evaluation team; what is needed to address the evaluation questions; and what is needed to ensure the scientific rigour of the evaluation

• Careful planning and continuous reflection are needed when trying out innovative methods not used previously

• Opportunities to feed findings back to stakeholders need to be built into the evaluation plan. Ideally, these should include opportunities during the evaluation process, for example when preliminary results from each method are available, and at the end of the evaluation, to obtain input on the integrated findings