BMC Medical Research Methodology BioMed Central Debate

Background The habitual "any other comments" general open question at the end of structured questionnaires has the potential to increase response rates, elaborate responses to closed questions, and allow respondents to identify new issues not captured in the closed questions. However, we believe that many researchers have collected such data and failed to analyse or present it. Discussion General open questions at the end of structured questionnaires can present a problem because of their uncomfortable status of being strictly neither qualitative nor quantitative data, the consequent lack of clarity around how to analyse and report them, and the time and expertise needed to do so. We suggest that the value of these questions can be optimised if researchers start with a clear understanding of the type of data they wish to generate from such a question, and employ an appropriate strategy when designing the study. The intention can be to generate depth data or 'stories' from purposively defined groups of respondents for qualitative analysis, or to produce quantifiable data, representative of the population sampled, as a 'safety net' to identify issues which might complement the closed questions. Summary We encourage researchers to consider developing a more strategic use of general open questions at the end of structured questionnaires. This may optimise the quality of the data and the analysis, reduce dilemmas regarding whether and how to analyse such data, and result in a more ethical approach to making best use of the data which respondents kindly provide.


Background
This paper is about ways of generating reliable knowledge in homeopathic medicine -though the ideas in it can be extended to other areas of medical inquiry. One caricature of contemporary homeopathic research would see it divided into two camps -university researchers in one and engaged homeopathic practitioners in the other. The former has traditionally focused on the double blind, placebo controlled, randomised controlled trial (RCT) as its principle means of investigation, whilst the later has focused on creating case reports and conducting homeopathic "provings" -also known as "homeopathic pathogenetic trials". [1] Its ready applicability to pharmaceutical interventions and its ability to deal with systematic bias has led the RCT to become the method for evaluative research in academic medicine. RCTs of homeopathy have proven inconclusive. Linde's meta-analysis of 1997 showed that, overall, trials of homeopathy indicated an effectiveness superior to placebo. [2] However the larger and more high quality trials were more likely to be negative and in the last two years a string of inconclusive trials have been published. [3][4][5][6] Some of the challenges of applying RCT designs to homeopathy have been described by other authors. [7,8] A central difficulty is that the placebo-controlled RCT has low external or "model" validity in relation to the practice of homeopathy in real-world contexts. For instance in the homeopathic management of on-going illness, the introduction of placebo medication affects the interpretation of progress at follow-up -has the practitioner chosen the wrong medicine or has the patient been allocated to the placebo arm? Also the homeopathic approach includes an in-depth consultation, aspects of which are potentially therapeutic and which are received by both arms in a trial.
Meanwhile practitioners, many of whom are qualified medical doctors, appear to see convincing therapeutic outcomes in their clinics. [9] For them the songbird of homeopathy sings often enough to keep them engaged in a demanding discipline. Within the community, therapeutic knowledge is conveyed primarily through written and seminar reports of these successful cases. These cases are one of the key sources of knowledge on which homeopathic practice is based. By contrast knowledge from RCTs is of little practical use to the homeopath.
The practice of basing therapy on knowledge of individual cases is not in vogue in mainstream medicine -except perhaps with respect to adverse event reporting. A widely used Evidence-Based Medicine text states that "because these studies lack comparison groups, they are usually only sufficient for hypothesis generation and thus point to the need for other studies". [10] We know an observational case-based epistemology was central to Hippocratic medicine and this tradition was revived by physicians such as Sydenham -a key instigator of what we know as modern clinical practice and famed for his ability as a natural historian. [11] By way of contrast cases are a central component of medical education -unsurprising since the results of all clinical epidemiology have to be interpreted in light of individual predicaments. [12] Getting research findings into practice is one of the central challenges of EBM. [13] In the case of homeopathy the practitioner would turn to a non-research journal such as "Homeopathic Links". The case reports in this journal, that adhere to guidelines for authors [14], are of as high a quality as any currently available. Such case reports serve the practitioner community well but, by even the most liberal criteria, can be considered only weakly scientific. Reports are anecdotal with the patient's story nested within the (normally affirmative) therapeutic story of the case within a larger story of homeopathy as an effective system. The "discourse" (i.e. the written/verbal expressions of the tacit assumptions of a particular community) is firmly homeopathic. This type of reporting has inherent limitations that reduce its value as a source of trustworthy knowledge. Some of these are listed in Table 1.
This paper opens for debate the idea of a middle way between the anecdotal informality of the case report and the simplifying assumptions of the RCT. It suggests methods that may still allow the songbird to be heard without the merits of that song being exaggerated. It suggests methods that can be adopted by both academic and practitioner-researchers on a variety of scales to produce results of value to both the sceptical enquirer and the homeopathic community.
This middle way consists of research designs that can be termed "formal case studies" (FCSs) to contrast them from the informal case reports above. The FCSs can employ the established methods of qualitative research to improve the rigour and sophistication of our approach to homeopathic cases. The remainder of this paper will discuss the process of designing formal case studies in homeopathy and ensuring the trustworthiness of their findings. Prior to this the case study as formal research method is introduced.

Case studies in context
Case studies are little used in contemporary medical research though they have been deemed "probably the most neglected of clinical research methods" by an author writing about Primary Care research in the UK. [15] This contrasts with fields such as education, business and governmental research where case studies are probably the commonest method in use. As mentioned above, Western medicine does have a long tradition of case-base learning stretching back to the astutely observing Greeks which was eclipsed by the rise of laboratory medicine in the nineteenth century and statistical inference in the twentieth. [16] What are the relative merits of the case-based approach? This question is well covered elsewhere -for instance Lukoff in a comprehensive overview reasons: "because case studies examine people in or close to real situations, and because they do not distort naturally occurring behavior through experimental manipulation and the setting up of artificial conditions, they usually offer better external validity than do controlled experiments.". [17] Recall that it was lack of external validity (the extent to which the study findings can be duly related to the real world) that was the main critique of the RCT. Practitioners treat individuals, with whom a host of variables may serve to frustrate the neat implementation of trial data derived from populations. The individualising nature of homeopathic treatment (where remedies are not typically prescribed on the basis of a diagnosis but on a study of the particular features of a case) makes a case-based enquiry essential to the generation of knowledge "within" homeopathy.
FCSs are grounded in the established practices of qualitative research -see Cassidy for a review of qualitative methods as applied to CAM. [18] The principles of qualitative research have been applied for decades in disciplines such as sociology, anthropology and education and have recently begun to find favour in medicine. [19,20] It applies itself principally to questions that require an indepth understanding of human decision-making, opinion, behaviour or experience. What qualitative research studies lack in scale they often make up for in depth or "thickness" of understanding in addressing issues that could not otherwise be tackled (see below and Table 2 for examples).
Yin, author of an influential textbook on case studies, defines three main types -the exploratory, descriptive and explanatory. [21] Exploratory and descriptive case studies do not offend the normal logic of epidemiology but understanding how case studies can be used to explain observations is more challenging for the medical reader. Explanatory case studies must begin with some sort of proposition -preferably grounded in established theory. These propositions are then tested by a variety of analytic tools, central to which is the search for rival explanations. This process leads, through a "replication logic", similar to that found in laboratory medicine, to a theoretical generalisation to similar contexts outside the study. This process is found in many other branches of academia -e.g. historical research, and can be termed "theoretical generalisation".

Other types of observational research
The formal case study is of course not the only method to straddle the gulf between RCTs and informal case reporting. A variety of observational studies have been published in homeopathy including clinical audits [23], prospective observational studies [24] and pragmatic RCTs. [25] (these are RCTs in which homeopathy is compared not to placebo but to a "standard" or alternative intervention). The role of non-RCT "outcomes" research in CAM has recently been reviewed. [26] Observational methods are typically deemed inferior to RCT designs though recent findings show that studying outcome using cohort or case-control designs (i.e. without placebo or randomisation) does not in fact lead to (the presumed) overestimation of treatment effect. Such designs are also less prone to creating paradoxical conclusions than RCTs. [22] So far this paper has argued for the need for formal approaches to case studies in homeopathy and provided an introduction to a method grounded in qualitative research. In the next sections the process of designing a FCS is described.

Designing a Formal Case Study (FCS)
Formal case studies can be designed and conducted by homeopathic practitioners though their credibility is increased with the inclusion in the design and implementation of neutral or sceptical colleagues. Though FCSs can meaningfully include a single case they are better designed to include a series of cases as the comparison of individual cases is an important part of the analytic proc- 1. Submitted by practitioners predisposed to view the therapeutic process through the theories of homeopathic medicine. 2. Rival explanations for changes in health status not systematically investigated 3. Objective evidence in the form of questionnaire responses, laboratory findings and reports from other carers not normally included. 4. Dependent on the accuracy of the notes and memory of the practitioner 5. Typically retrospective allowing for selection of best results -not therefore indicative of the nature of practitioner's practice as a whole. 6. Duration and detail of follow-up frequently insufficient to be sure of depth and longevity of any therapeutic response. Unchanged symptoms not usually included in follow-up reports.
ess. The design process can be divided into several stages as follows.

Deciding on the focus of the study
One of the strengths of the FCS is that it can address a diversity of issues -including but not limited to the assessment of outcome. Many such issues are "internal" to homeopathy and would not typically find publication in a journal such as BioMed Central but which would be appropriate for the homeopathy research journal "Homeopathy". A typology of potential FCS research questions is provided in Table 2.
Some of these questions are of wider interest. For instance, as Swayne has highlighted, homeopathic consultations contain some of the richest "pathographies" (verbal descriptions of the evolution of disease) in medicine. [27] "Narrative-Based Medicine" is now widely discussed in the UK and is concerned, amongst other things, with giving voice to what it is really like to live with given diagnoses. [28] Interviews with homeopathic patients could potentially be used for on-line databases such as DIPEx (the Database of Individual Patient Experience). [29] Perhaps too the consulting style of homeopathic doctors, with which patients often express satisfaction, has something to teach the medical profession in general. [30] Sampling in the FCS FCS design is almost certainly prospective whereas most case reports are put together retrospectively when the patient has been seen to respond well. There are many possible "frames" for prospective sampling including each consecutively referred patient, every 10 th patient or every patient presenting with a particular diagnosis. The latter would obviously be necessary for the researcher interested in the effectiveness of homeopathy for a given condition but for research into, say, the nature of homeopathic aggravations, it would not be particularly important to study patients with the same diagnosis. Sampling for cases responding to a particular remedy would be more difficult to do prospectively -unless data were collected on a great many cases and then grouped according to remedy. The utility of collaborative efforts in FCSs is apparent.

Planning Data Collection
Case studies are criticised when they rely on a database that depends on the notes and memory of the practitioner -as is normally the case in informal case reports. FCSs are distinguished by the range of data sources that are ulitised to triangulate study findings. Triangulation is a term from qualitative research and is one of the methods used to increase the trustworthiness of study conclusions. If therapeutic change reported by a practitioner is confirmed by the patient's illness journal, GP, blood results, numeric outcome measures, spouse and by independent interview, then it can be said to have occurred beyond reasonable doubt. Of course that still leaves open the question of how Obviously not all of these would be employed in a given study -but the more formal the enquiry the more need there is to use multiple sources of information to contribute to the internal validity of the conclusions. Comparison of these data sources would be a valid study in itself. How do practitioner notes compare to verbatim transcripts? How does family member testimony compare to those of the patient -anecdotal evidence suggests they may be at variance? What light does a pre-treatment questionnaire shed on the therapeutic process?
The decision to make verbatim transcripts of consultations has a major bearing on the cost and labour-intensity of the study -as one hour of consultation may take eight hours to accurately transcribe and as long again to analyse. Though this paper is focusing on qualitative methods there is no a priori reason why these cannot be combined with quantitative approaches using a wide spectrum of generic and condition-specific numeric outcome measures such as EQ-5D and MYMOP. [31] The comparison of these sources can produce interesting results. [32] The timing of the use of the various data sources needs to be carefully articulated in a study protocol and progress logged in a database. The art of the FCS is to standardise data collection in all areas that will not impinge negatively on the normal homeopathic process. Rigor at this stage will effect the trustworthiness of the conclusions through allowing consistent cross-comparison of cases. So for instance in a case series you might decide that each participant will have the same number of consultations.

Data Analysis in FCSs
Time-consuming and intellectually challenging, qualitative data analysis (QDA) is the process by which the study data are interrogated for information relevant to the questions under consideration. The thoroughness of the analysis depends on the context of the study but should ideally include examination of all data. Most qualitative researchers make use of sophisticated software packages such as Atlas.ti and NUDIST to aid data handling of textual data such as interview or consultation transcripts.
Taking the example of consultation transcripts, the initial analysis involves taking aspects of the data and ascribing to them some sort of meaning -one might for instance devise a system for identifying different types of questioning by the practitioner: open, closed, hypothetical etc. Or one might identify any examples of "homeopathic aggravation" (the worsening of the presenting symptoms under treatment). This process is called "coding". For a descriptive case study this may be enough. Often however there is a proposition driving the inquiry. For instance the researcher might wonder if talking about hurtful events is part of how homeopathy functions and set up their FCS as a test of Pennebaker's "disclosure theory". [33] The researcher would look at outcome in relation to disclosure with particular attention to "deviant cases" where one might, for instance, see recovery without disclosure. [34] For a fuller understanding of analysis the reader is referred to Yin. [21] For qualitative methods in general Mason [35] is a good starting point with texts such as Bryman and Burgess [36] taking a broader view. As in the wider world of academia, conclusions are drawn through empirically based reasoning.  1. Previous medical records of the patient 2. Pre-treatment questionnaires 3. Written/typed notes of the practitioner * 4. Verbatim transcripts of consultations 5. Audio and video tapes of consultations 6. Generic numeric outcome scores (eg EQ-5D or GHQ) 7. Condition-specific outcome scores (e.g. BDI, IBS-QoL) 8. Physical and laboratory measurements (eg bp and ESR) 9. Journals written by patient through therapeutic process 10. Data from interviews of patient by an independent researcher 11. Testimonies of family members 12. Testimonies of other carers including GP and hospital specialist 13. Details of all concurrent medications 14. Other -eg paintings or poems by patient 15. Notes from "supervision" of homeopathic prescribing 16. Information from the homeopathic literature (eg Materia medica)

* -this is often the only source used in informal case reports
In this section of the paper a broad framework has been proposed for FCSs in homeopathy in terms of research questions, data collection and analysis. The next section describes how academic rigor can be applied to the whole process.

Means of ensuring trustworthiness of FCS findings
One of the main criticisms of case study research is that it is interpretative -concerned with the subjective appreciation of meaning. It is unlikely that, presented with the same data, two researchers would reach exactly the same conclusions. The discipline has evolved several important ways of responding to this inescapable problem (in addition to the triangulation mentioned above). See Bloor [37] and Barbour [38] for a critical appraisal of these strategies and Lukoff again for a review in the CAM context. [17]

Groundedness
The importance of groundedness extends to all QDA regardless of whether or not the research espouses a formal "grounded theory" approach. [39] It means that conclusions can be shown to be firmly rooted in the data and not the offspring of the researcher's/ practitioner's imagination -and also explains the liberal use of quotations in the presentation of QDA. Yin borrows the legal metaphor of a "chain of evidence" signalling that the reasoning process from initial questions to final conclusions should be clearly explicated -in the homeopathic context that might include clear descriptions of the rationales for various medication choices.

Exploring alternative explanations
This is a particularly important component of FCSs in the homeopathic context as practitioner-researchers will inevitably work through assumptions resting on their belief in homeopathic theory. FCS researchers need to embrace a range of rival alternative theories for change in patient health status including chance, natural regression, therapeutic consulting and other therapies and show that these have been evaluated in their analyses.

Peer-review and respondent validation
In peer-review, analysis is reviewed either by another practitioner (as happens in case-supervision) or by a worker in another field altogether. Respondent validation refers to review of study findings by participants in the research. This can throw up ethical problems in the health context depending on how consensual the process has been. A simple statement by a patient that they deem the account of therapy to be accurate would in itself be a significant validation. Obtaining the account of a non-professional friend of the patient -what could be termed the "significant other report" is an additional sophistication.

Reflexivity
Qualitative research is distinctive in its acceptance of the fact that it is not possible to be a truly objective researcher. As this case study researcher comments "it is better to recognise and be explicit about possible sources of influence, and to make allowances for them than it is to pretend that such influences do not exist or to regard them as unwanted sources of "bias" and attempt the impossible task of eliminating them altogether." [40] FCS researchers need to be aware of (and try and explain in presentation) the impact they are having on the collection and analysis of data. Such relativism would be laudable all research communities with a tendency to absolutism -but the challenge of reflexivity should not be underestimated. [41] In some circumstances the researcher (?practitioner) can become the focus of the study through the techniques of autoethnography. [42] In this context it is interesting to note the many parallels between the skills needed for homeopathic practice, such as interviewing and the analysis of meaning, and those required for qualitative research. [43].

Writing up the FCS
The intended style and eventual forum for the write-up should be considered in the initial design process as it will effect how the data is generated. There is no reason why an astutely observed, reflexive and theoretically rich account of a single case could not make a significant contribution. Ideally such a case report should be more than purely descriptive but set out to make a general point. For instance in one of the few homeopathic cases written up in the non-homeopathic literature (excluding those solely reporting adverse events) the paper seeks to illustrate how homeopathic medicine was used alongside a high technology intervention in palliative care. [44] Single case designs are also appropriate when writing for a purely homeopathic audience and guidelines for authors have recently been published by the journal "Homeopathy". [45] Whilst generally not adopting the formal approach advocated here, this journal has published increasing numbers of case-based articles.
Most commonly FCSs would comprise a series of cases opening up several publication options. One choice is to present the narratives of individual cases around a particular theme with short introductory and concluding sections to bracket the whole work -see for example Geraghty. [46] Another is to use the report to analyse various themes quoting selectively across the cases. For instance this might be a style used in a book examining aspects of the homeopathic consultation process. This style has the disadvantage of breaking the narrative coherence of the cases or requiring very short case descriptions. Such a presentation would contain considerable crosscase comparison.
Yin defines some criteria for excellence in case reports. They must tackle a significant topic, show evidence of including a reasonably complete account of the topic (see Box 2 for types of evidence in homeopathy), consider alternative perspectives and be written in an engaging style.

The ethics of case study research
The case study researcher/practitioner faces particular ethical challenges including the potential for the research process to compromise the therapeutic process -simply perhaps by dividing the worker's attention between the two activities. The researcher, particularly the practitionerresearcher, has to be committed to an honest exposition of the study findings even if these do not reflect favourably on the effectiveness of the practice. Where a study reflects on the activities of a clinical team -the implications of "revealing" an under-performing colleague need to be thought about in advance. Another major issue is the anonymisation of so much individualised information in a way that does not distort its meaning -you can't change a plumber into a banker. The ethical aspects of qualitative research have been thoroughly explored. [47]

FCS in homeopathy in Bristol, UK
Of the many interesting case-based studies cited, none has substantially employed formal qualitative methods. In Bristol we are conducting a study to apply these formal methods to the study of the "players, process and outcome" of the homeopathic approach in 18 patients attending the Bristol Homeopathic Hospital. All patients treated at this hospital receive a "package of care" comprising a new patient appointment of roughly one hour's duration and four follow-up consultations each lasting around 15 minutes. Of the 18 patients, six are living with IBS, six with Chronic Fatigue Syndrome and six are children (aged 5-16) with childhood eczema. There are three doctors in the study -each seeing two patients with each of the three index conditions. Patients are being interviewed before and after the package of care and individual consultations are being recorded and transcribed. In addition outcome questionnaires, medical notes, conventional medication use and "significant other reporting" are being incorporated as primary data. The purpose of this study is to understand the expectations of patients referred for homeopathy, detail potential "active ingredients" of the homeopathic process (using the MRC framework for the analysis of complex interventions). [48] and specify the outcomes of treatment from a patient-centred perspective. The study will also examine some of the theoretical suppositions internal to homeopathy including the nature of healing responses. The study hopes to report early in 2005.

How FCS links with other research methods in homeopathy
The FCS can be an important tool for the development of clinical trials -this is an explicit recommendation of the MRC framework. Once we understand potential "active ingredients" we can start to think of ways of testing these against appropriate controls. For instance in the pragmatic controlled trial, homeopathic care is tested against alternative or standard care -as opposed to against placebo. The FCS would also be ideally nested within a prospective outcome study, [26] where a given proportion of patients would have their cases studied in greater depth. Because the FCS examines both process and outcome with a particular search for "context effects" it is also applicable to the emerging area of "whole systems research" in which, for instance, the "patient-practitioner dyad must become a major focus for the understanding of healing, including what both practitioner and patient bring to the encounter". [49] Summary This article raises for debate the idea that research in homeopathic medicine could be usefully progressed by applying formal qualitative research methods to the study of homeopathic cases. It identifies the limitations, with respect to homeopathy, of both placebo-controlled trials and informal case reports and proposes the formal case study (FCS) as a potential middle way. RCTs are limited primarily by the problems they have with external or "model" validity with respect to a complex intervention like homeopathy. For instance it is probably a mistake to conceptualise homeopathy as being equivalent to conventional drug as the "drug" selection process is itself a potential therapeutic intervention received by both arms in a trial. Informal case reports serve a useful purpose within the discourse of homeopathy but are unconvincing from even a liberal scientific perspective, being retrospective, memory dependent, partial and lacking in external confirmation (see Table 1) The FCS is founded in an historical context of case-based research that extends back to antiquity. It allows the naturalistic description of clinical cases and encourages the recording of individuality and anomaly. The FCS can draw on the established practices of qualitative research to substantially improve the trustworthiness of study findings and to form a bridge between homeopathy and the wider research community. The FCS and be pursued by university and practice-based researchers -or ideally the two in tandem. As discussed, homeopathic practitioners may have a natural aptitude for such studies.
Anyone attempting a FCS should first establish a focus for the study, preferably with reference to existing literature and theory. Inquiry can focus on questions both "internal" and "external" to homeopathic discourse (see Table 2). Examples of the latter include issues around patient narratives, satisfaction with care, and subjectively perceived outcome. Studies should be prospective and data generated from a variety of sources to ensure adequate triangulation especially with respect to outcome (see Table 3). Though time-intensive, audio and video recording of consultations with transcription into text, offers the greatest opportunity for in-depth analysis.
Analysis in FCS should be grounded in the primary data and seek to test and exemplify established concepts. Central to analysis is the comparison of data from different cases and the active search for cases which do not conform with a developing theory. In particular, alternative explanations for changes in health status should be systematically explored. The trustworthiness of study findings can be increased by the application of techniques from qualitative research including groundedness, triangulation, peer and respondent validation and reflexivity.
Triangulation within FCSs is reminiscent of the arguments put forward by Jonas concerning CAM research as a whole. Jonas eschews the conventional "gold standard" superiority of the RCT and argues instead for an "evidence house". [50] This has many rooms representing different types of evidence meeting the complementary needs of diverse stakeholders such as patients, practitioners, basic scientists and public agencies. As with RCTs, FCSs are not in themselves complete or sufficient.
Can the caged bird sing? Any scientific process places constraints on the system under consideration. In this article a method is set out which can be at once systematic and naturalistic in its handling of a delicate person-centred therapy. As such it offers an important new avenue for the scientific exploration of homeopathy.