Negative pressure wound therapy: Potential publication bias caused by lack of access to unpublished study results data

Conservative estimates report that about 1% of the population in Western countries is affected by chronic wounds [1], with a much higher prevalence rate in inpatient and nursing facilities [2]. Besides presenting a significant risk factor for complications such as infection and amputation, chronic wounds also lead to marked impairment of patients' quality of life [3, 4].

Negative pressure wound therapy (NPWT) consists of an open-cell foam dressing covered with an adhesive drape. The dressing is connected to a vacuum pump that creates and maintains a subatmospheric pressure [5]. The most commonly used NPWT device is the Vacuum Assisted Closure (V.A.C.^®) device.

The Federal Joint Committee (Gemeinsame Bundesausschuss), the decision-making body of the self-administration of the German health care service, commissioned the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG) to conduct a systematic review on the efficacy and safety of NPWT versus conventional wound therapy in patients with acute or chronic wounds. The full review was published in German in March 2006, and an English-language article has recently been published [6, 7]. An update of the systematic review, a rapid report, was published in German in January 2007 [8].

Although NPWT is widely applied, particularly for chronic wounds, at the time of the review only a few relevant randomised controlled trials (RCTs) were available; these were, moreover, of poor quality [9]. Previous reviews on NPWT have not provided clear evidence of the superiority of NPWT over conventional wound therapy. For example, a review published by the Canadian Medical Advisory Secretariat in 2006 concluded that "Based on the evidence to date, the clinical effectiveness of NPWT to heal wounds is unclear" [10].

Previous reviews on medical interventions have shown that conclusions on efficacy and safety based on published data alone may no longer hold after consideration of unpublished data, which may reverse favourable risk-benefit profiles and attenuate treatment effects of an apparently superior intervention [11–13]. Within the framework of the systematic review on NPWT, our main objective was therefore not only to identify published, but also unpublished completed or discontinued RCTs in order to gain as complete an overview as possible of the evidence on NPWT. Further aims were to locate ongoing RCTs for potential consideration in future updates, and to determine the type of sponsorship of all eligible trials.

We then screened online trial registers (Table 1) and the US Food and Drug Administration website, and contacted German authorities, requesting information on all available RCTs on NPWT. Prior to the start of the review, the Federal Joint Committee had asked experts to provide relevant literature; these citations were also obtained by IQWiG as part of the commission. Moreover, the reference lists of retrieved original articles and systematic reviews, as well as congress proceedings, were handsearched.

Two manufacturers of NPWT devices were identified: Kinetic Concepts Inc. (KCI, San Antonio, Texas, USA) markets the V.A.C.^® device, and BlueSky Medical Group Inc. (BSM, Carlsbad, California, USA) markets the Versatile 1 Wound Vacuum System^®. Both were asked to provide information on the study status and publication status of sponsored trials, as well as on methods and results. Finally, we contacted the authors of conference abstracts by e-mail and by post. All contact details of the relevant institutions were checked beforehand on the corresponding websites. Manufacturers and authors were informed that IQWiG does not accept "commercial in confidence" data and publishes all data contributing to a systematic review [15].

All languages were included, as long as a title was available in English. If the title indicated a potential relevance of the study, the corresponding article was obtained and translated. Only RCTs comparing NPWT versus conventional therapy were eligible. Trials were considered nonrandomised if concealment of allocation to treatment groups was classified as "inadequate" [16]. The intervention was classified as NPWT if a medical device system identical or comparable to the V.A.C.^® Therapy System was used.

Results of trials that were not available as full-text publications were only considered in the review if detailed data were available (e.g. from a clinical study report or a manuscript in press). Results data from conference abstracts were not considered.

All steps of the literature screening were performed by two reviewers independently of one another. Any disagreements were resolved by discussion. The titles and abstracts of the retrieved documents were screened to exclude citations that were clearly irrelevant. The full texts of the remaining potentially relevant articles were then screened to identify RCTs that fulfilled the inclusion criteria stated above.

With the help of the information obtained, the study status of the RCTs identified was classified as "completed", "discontinued", "ongoing" or "unclear". The terminology used in the literature to classify unpublished data or so-called "grey literature" is inconsistent [12]. We classified the publication status of completed or discontinued trials as "published" if a full-text paper on final study results (completed trials) or interim results (discontinued trials) was identified in the literature search, and "unpublished" if this was not the case. In addition, for ongoing trials it was indicated whether interim results were available or not. The type of sponsorship was also noted for all trials.

Only a summary of results on the quality assessment and outcomes of the RCTs included in the IQWiG systematic review and rapid report is presented here, as the main focus of this paper was to identify unpublished RCTs.

The main objective of this paper was to identify unpublished RCTs on NPWT within the framework of a systematic review. An RCT was classified as "unpublished" if no full-text paper on final study results (completed trials) or interim results (discontinued trials) was available.

An extensive search strategy was employed that included handsearching of retrievals, as well as contacting manufacturers and authors of publications. The sensitivity of bibliographic database searching for RCTs is reported to be unsatisfactory [46], and multi-source searching has been recommended to retrieve all available RCTs [47]. In this context, the usefulness of handsearching and contacting experts has been demonstrated [48, 49]. The extensive search strategy was also useful to detect ongoing RCTs, of which one [22] was subsequently added to the evidence base.

A total of 28 RCTs referring to at least 2755 patients were identified. The publications on completed or discontinued RCTs reported data on less than a third of patients planned or analysed. Less than half of the conference abstracts on these trials were later published as full-text publications; similar results were found in a review by Scherer et al, who found a full-text publication rate for abstracts describing RCTs or controlled clinical trials of about 60% [50].

On the basis of results of published data alone, the IQWiG review and rapid report concluded that although there was some indication that NPWT may improve wound healing, the body of evidence available was insufficient to clearly prove an additional clinical benefit of NPWT. This finding is in line with findings from previous reviews.

Regarding the discontinued unpublished trials, according to the manufacturer the main reason for discontinuation was insufficient patient numbers. No results data for these trials were available. Regarding the completed unpublished trials, the authors of conference abstracts did not respond to IQWiG's request for information. Therefore, unpublished results data could not be considered in the IQWiG review and consequently the impact of unpublished data on the conclusions of reviews based on published data could not be assessed.

Multi-source comprehensive searches identify unpublished clinical trial data. However, lack of access to unpublished study results data raises doubts about the completeness of the evidence base on NPWT. The implementation of regulations such as prospective mandatory trial registration and the obligation to publish all results is needed to ensure that independent researchers have access to all outcomes of completed or discontinued clinical trials.