How can the differences be explained?
We interpret the main finding in our comparison above as an indication that the attitudes of the samples are comparable, and that the difference is not one of essence, but rather one of degree. This difference between the focus groups and the survey could reflect methodological flaws tied to these two particular studies such as the representativeness of the small and self-selected focus group sample. More detailed discussions of the strengths and limitations of the two studies can be found in the corresponding publications [22, 24], but a few key problems need to be mentioned here:
The survey is based on a representative sample with an acceptable response rate, and the sample in the focus group study is quite similar to the sample in the survey according to some observable variables. Thus both survey and focus group participants share specific background characteristics with the general GP population. While the survey respondents may be more motivated then the non-respondents, this motivation is not necessarily tied to the questions about guidelines as this only constituted a small part of the panel study. The focus group participants, on the other hand, may be seen as less representative than the survey participants because of the lower number of participants and stronger issues of self-selection. Although the focus group participants were similar to the survey participants in terms of certain observable background characteristics, we know less about their characteristics in terms of motivation, personalities and general attitudes, and it could be argued that such a self-selected sample is often made up of individuals with a message. However, in this case, we invited the groups via the group leaders, who are responsible for setting up a programme for each group session of the educational group. We therefore expect that the group participants may have had varying levels of interest in our endeavour. As for the group leaders, they may have had especially strong opinions about the subject but could also simply have been motivated by the offer to be spared the work of planning a group session. Moreover, an advantage of focus groups is the opportunity to ask participants to explain their motivations. Our initial questions regarding motivation led us to believe that the group leaders were more motivated by a general interest in improving practice than in guidelines in particular.
We have also considered how the researchers’ may have influenced the results of the focus group study. When scrutinising the transcripts again in light of the survey results we could not see that the researcher tended to ask leading questions or probe for negative views. Instead, in our opinion, the researcher held a rather low profile during the group discussions. However, due to the fact that the researchers enjoy more flexibility when interpreting qualitative material than they do when interpreting quantitative data, emotional utterings or detailed stories from a few convincing informants may influence the researchers to draw too strong or even distorted conclusions. While we recognise our limited ability to assess our own research, we did take the recommended precautions in the analysis (e.g. recording the discussions and involving another researcher who had not participated in data collection). Also, in the re-analysis done for these methodological reflections, we introduced an external researcher (CG), who had neither been involved in data collection or analysis of the two primary studies.
It is also worth noting that although the survey depicts more positive attitudes towards guidelines, the barriers identified in the focus groups are strongly supported in the survey, and the relative importance of the barriers is the same in the two studies. In addition, the findings in both studies are supported by other studies from other researchers.
We therefore suggest that both studies are basically methodologically sound and that the discrepancy in results therefore is not primarily caused by contextual methodological limitations.
An alternative explanation for these differences could be that they are an indication of a general methodological phenomenon: that the two data collection methods tend to bring forward different attitudes. Below we refer to empirical research which suggests that problematic issues are under-communicated in surveys. Our experiences from mixing these two methods suggest that focus groups, in contrast, appear to overstate problematic issues. Does existing literature on survey and focus group methodology support this hypothesis?
Do surveys paint too bright a picture?
As mentioned above, the literature indicates that survey respondents tend to understate problems and negative feelings . Survey participants’ desire to please researchers in surveys has been documented in a range of studies under different headings such as courtesy bias[2, 27] and socially desirable responses. Researchers using survey methods have sometimes made use of observational data to compare people’s actual behaviour with the behaviour they have described in the survey. With regard to our own research theme, various studies have documented that survey respondents predominantly answer positively to general questions about guideline adherence , and exaggerate their own guideline adherence rates .
It may be argued that these general characteristics, or biases, of surveys alone could explain the discrepancy in our results. However, considering the apparently striking difference in the findings of our studies and the fact that we have very little research-based knowledge about the typical workings of focus groups, we believe it is possible that the focus group method has its own biases.
Do focus groups paint too gloomy a picture?
It is of course possible that focus groups also generate courtesy bias, i.e. that participants understate problems and over-report socially desirable behaviour. This has been discussed as a hypothetical problem , but empirical verification is lacking. Conversely, it has been argued that the fact that the researcher(s) is outnumbered in focus groups leads to a conformity bias amongst the participants rather than an urge to please the researcher(s) [20, 31, 32], but, again, there is a lack of empirical data to support this hypothesis.
Faced with a lack of direct evidence with which to better understand focus group processes specifically, some researchers have turned to the field of small group dynamics . Within this field, it has been demonstrated that we have a tendency to hide differences of opinion when we participate in group sessions , a phenomenon that is incorporated in the term group conformity. As focus groups are supposed to be well suited to capture the common norms of a social group [31, 34], group conformity may in fact be employed as an important tool in focus groups . Researchers are commonly advised to place homogenous participants in the same focus group, which could reinforce such tendencies . Denzin and Lincoln suggest that in focus groups, “collective identities are consolidated” (p904). Robson claims that focus groups lead to consensus and that deviant views tend to be “weeded out” (p284). In much the same way, Albrecht argues that focus group participants “discuss an issue and offer a unified voice in presenting their opinions to the researcher”  (p53). In contrast, Morgan and Krueger argue that focus groups are not comparable to natural groups in that they are monitored and that, according to their experiences, the tendency to conformity in focus groups is merely a myth . This view is partly supported by one of the few methodological reports from a focus group study, which offers examples of how “brave individuals” challenge the dominant view of the group . But still, data is lacking as focus group discussions have not been subjected to empirical research to the extent that other group processes have.
Another interesting finding in empirical research regarding group behaviour is the phenomenon that social groups reinforce attitudinal tendencies and thus lead the group towards extreme views  or group polarization[10, 40]. Hogg and Turner  argue that group polarization is the effect of conformity to a norm that defines the participants’ own group in contrast to other groups. A related theme is exaggeration bias where group dynamics lead to a “bandwagon effect”, with group members endorsing more extreme ideas than they would express individually . This view is supported by more recent research where it is noted that group participation may lead to a ”movement to more extreme positions”  (p46).
With regard to focus groups, some authors  have argued that the presence of other participants in the group may prevent participants from exaggerating. However, inspired by small group research, Sussman et al.  compared focus group discussions with participants’ responses in a survey before and after the focus groups and found that participants acquired more extreme attitudes after discussing the relevant subject in a focus group. The authors note that there seems to be an amplification of the prevalent group norms, or a “polarization effect”.
But could these group conformity and polarization effects take the form of a negative attitude bias in focus group studies? After carrying out a focus group study of school students’ views of their science classes, Watts and Ebbutt  suggested that focus groups encourage more critical comments than for example individual interviews: “Many people, and our youngsters may be no exception, once set upon a critical path, enjoy the opportunity of a collective 'moan session” (p32). The authors then go on to describe these group discussions as an “infectious downward spiral of shaded awfulness.” Watts and Ebbut’s paper is widely referred to, and several authors have used their assertion to explain an excess of negative comments in their focus groups. However, we have found no study involving empirical comparisons between methods that could support this claim.
A related mechanism could be that focus groups may be experienced as particularly well suited for forwarding complaints. Participants are offered time and encouragement to explain their attitudes and personal experiences regarding a problem area in detail to external researchers who, in addition, may be connected to the relevant authorities/policy makers, as we were in this focus group study. This could make the focus group a good opportunity for those who want to forward a message of complaint about a difficult situation or an unjust practice. It is therefore essential that focus group researchers report and reflect upon how participants are recruited, what motivated them to participate and that the researchers discuss and reflect upon who they represent towards the participants. In our particular comparison, we have taken account of these issues and, as mentioned, found no strong reason to suspect that the participants were driven by a wish to complain to start out with. We suggest instead that the participants’ complaints may be a characteristic of the group process rather than a reflection of their original motivation to participate in the focus groups.
It has been argued that the potential of mixed methods is not realised in practice, e.g. that data from randomised controlled trials are seldom integrated with attached qualitative data . Could this be due to difficulties in comparing findings from different types of studies? This question is related to the basic debate about whether different methods offer completely different types of data, which therefore are difficult to compare, or whether they instead can yield comparable and thus, either supportive or conflicting data. Many qualitative researchers would argue in favour of the first postulation; that the questions asked and the findings drawn from focus group studies and surveys are of different natures and thus incomparable. If we consider the epistemology of the first postulation to be a relativist stance, and the second a positivist stance, we would argue for the pragmatic middle position, of subtle realism. We acknowledge that one research study will convey one of multiple existing perspectives, but argue that all perspectives are not equally important. While there is no eternally true, perfect story, some stories are less partial, less distorted and more relevant than others . One consequence of this view of science is that we attempt to select the methods most appropriate to the research aim. In this study, our aim was to find out whether the views expressed by focus group participants were typical of the general GP population. Here, the decision to compare focus group and survey data was a direct result of the research question and on how we planned to use the data. The questions in the survey were designed as postulations drawn directly from the summarised postulations that came up in the focus groups as we found that it was relevant and possible to check the distribution of these attitudes through the survey. Hence, the findings in the two studies are of the same type, and comparable in that respect. A different question is whether one would expect findings stemming from a small convenience sample to be reproduced in a broad, representative sample. In this case, the details of the barriers to guideline adherence were to a large extent confirmed in the survey, whereas the main impression or finding; the scepticism of governmental guidelines, was not confirmed. Adding this to other anecdotal observations from our work with focus groups, we found it reasonable to at least pose the question of a possible general phenomenon in focus groups.
Focus group methodology is an understudied area and our hypothesis rests mainly on only one empirical study and on inferences from research into small group dynamics. We are aware that knowledge about other types of small groups is not necessarily valid for focus groups. While extensive research into survey methodology suggests that surveys lead to overly positive findings, it is possible that our hypothesis regarding focus groups is false and that this method is not particularly liable to either overly positive or overly negative data. We have suggested alternative explanations; e.g. systematic tendencies regarding motivation among participants or researchers in FG studies, but still our hypothesis is plausible and may form part of the hitherto unexplored methodological characteristics of FGs.
Conclusive remarks and way forward
In our original presentation of our own survey findings , we made the reservation that people generally respond more positively in surveys than observations of their actions indicate. However, when presenting the focus group data, we did not discuss the possibility that the focus groups could have given an overly negative impression. Hereafter, we may want to consider making such reservations in focus group studies.
Methodological studies of questionnaire surveys have shed light on respondent behaviour; knowledge which is actively used by researchers when designing surveys and interpreting results. Methodological studies of focus groups could help researchers in similar ways, and could possibly facilitate integration of findings from qualitative and quantitative parts of mixed methods studies. Methodological studies of focus groups could include, for example, experimental studies that explore how focus groups influence participants’ attitudes; studies where actual behaviour and behaviour as reported in focus groups is compared; or, as Hyde et al. (2005) suggest, studies where individual participants are asked in post-session questionnaires to validate the group discussion. In addition, we encourage researchers of mixed method studies to explore whether there are systematic differences between findings produced by the survey and the focus group study that may be related to distinctive characteristics of the methods involved.