Table 2 HSR systematic reviews that have compared literature published literature with grey/unpublished literature

Maglione, 2002 [22] (Immunization Program)

Effectiveness of mass mailings to increase utilization of influenza vaccine among Medicare beneficiaries

Search of the Medicare Peer Review Organization Health Care Quality Improvement Project database

Six controlled trials were identified. Only one (earliest) trial reporting modest but statistically significant improvement in vaccination rate (2–8% depending on the format of the letter and location of the study) was published. Five subsequent trials which found smaller, clinically trivial improvement in vaccination rate of no more than 2% remained unpublished

The review only included a small number of trials identified from a single study registry and targeting a specific US population

Batt,2004 [20] (Immunization program)

Costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle income countries

Hand searches in institutional documentation centres including WHO and USAID; interviews with 28 international experts; search of grey literature databases; searches of the internet, conference proceedings and webpages of pertinent organizations

Quality of data on effect and cost-effectiveness was similar between published and grey literature, but the quality of costing data was poorer in grey literature. Inclusion of grey literature doubled the quantity of available evidence. Interventions examined in the grey literature were more up to date, associated with more complex interventions aimed at health systems and better represented west Africa and the Middle East. Conclusions drawn from the two sets of literature therefore differed

Reviewed grey literature was mainly derived from international organizations with little coverage of national governments. Searches were limited to English keywords

Fang, 2007 [21] (Organizational studies)

Relationships between organizational culture, organizational climate, and nurse’s job satisfaction and turnover

Extensive search of 35 databases, “footnote chasing”, and searching by author

Of the nine associations for which findings were compared between published articles and unpublished doctoral dissertations, significant differences were found for three of them: association between passive/defensive culture and job satisfaction; global climate and job satisfaction; and reward orientation climate and job satisfaction. All the differences were related to magnitude rather than direction of the estimated association

Grey literature was limited to doctoral dissertations. The number of studies was very small for some of the comparisons; in some cases only one published or unpublished study was available