Duration of the common cold and similar continuous outcomes should be analyzed on the relative scale: a case study of two zinc lozenge trials

In this study, the “absolute scale” indicates comparison on the scale of the original measurements such as days in the case of common cold duration. The “relative scale” indicates comparison with a placebo group level normalized to 1.0 or 100%.

The relative scale has been used for decades in analysing binary data in epidemiology. Relative risk (RR) allows the generalization of effects to different population groups, such as the 2-fold increase in total mortality and the 10-fold increase in lung cancer risk with smoking [1, 2]. Meta-analyses of binary outcomes on the relative scale have led to less heterogeneity than analyses on the absolute scale (risk difference) [3]. In addition, the relative scale cannot yield negative predicted values, whereas the absolute scale can.

In contrast, there has been a long tradition of carrying out meta-analyses of continuous outcomes on the absolute (original measurement) scale as the mean difference (MD), or using the standard deviation (SD) as the unit of the scale, which leads to the standardized mean difference (SMD) scale. Both of these approaches are available as options in popular meta-analysis software such as the RevMan program of the Cochrane collaboration [4].

The biological rationale for using the relative scale in the analysis of binary outcomes is that it adjusts for baseline variations. However, similar baseline variations can occur in continuous outcomes. For example, over 100 viruses cause the common cold, and the severity and duration of symptoms vary by virus. Since the distribution of viruses varies and different operational common cold definitions have been used in controlled trials, a substantial variation in the average untreated (placebo group) common cold duration is to be expected between trials. Since analysing the effect of treatment on the relative scale would partly adjust for such baseline variations between the placebo groups, the relative effect of a treatment on common cold duration might be more widely generalized than the absolute effect.

Friedrich et al. compared the absolute and relative scales in a series of 143 meta-analyses of continuous outcomes, and found less heterogeneity when the analysis was carried out on the relative scale [5]. They introduced the term Ratio of Means (RoM) to describe the calculation of the relative effect, and used a Taylor series-based approximation to calculate the SD for the RoM [5–7].

In our 2004 Cochrane review on vitamin C and the common cold, we used the relative scale, dividing the mean and SD of the cold durations of the study groups by the placebo group mean [8]. This approach is transparent and the SDs and the numbers of participants are apparent in the forest plots constructed with the standard programs. The effect estimate is identical with the RoM, yet the SD differs from that calculated by the Taylor series-based formula [5].

In epidemiology, the term RR is well established and the effects are often so large that the RR values are useful in communication. However, the treatment effects are often rather small and, instead of an RoM value, it may be more practical to consider treatment effects in percentages. Obviously, it is mathematically identical to state either that a treatment has an effect of RoM = 0.75 or that the treatment leads to a 25% reduction in the outcome. In this study, percentages are mainly used since they are familiar to most people.

If the relative scale is superior for some continuous outcomes compared with the absolute (i.e., MD) and the SMD scales, then the lack of the option of calculating the relative effect in popular meta-analysis packages would have led to suboptimal analysis of such outcomes. Although the calculation of the RoM estimates is simple on a spreadsheet, many meta-analysts do not consider approaches outside of the options provided by the standard programs. Furthermore, poorly selected measures of treatment effect can also hamper the communication of results to physicians and patients [5].

In considering the preference for the relative or the absolute scale, a question of fundamental importance is which of them is more consistent with the empirical data.

The approach of this study was to examine the predicted effects of zinc lozenges on the placebo group common cold durations by the absolute effect (in days) against the relative effect (in percentages). If the relative effect estimate is better, its application to the observed placebo group distribution would lead to a distribution of cold durations that is closer to the zinc lozenge group distribution. This kind of approach is informative only if the trial has demonstrated a significant difference between the treatment groups. This study analyses 2 trials that found benefit from zinc lozenges and for which individual patient data (IPD) were available [9, 10].

This comparison of the absolute scale against the relative scale in the analysis of the treatment effects on common cold duration focused on 2 randomized trials; the zinc gluconate lozenge trial by Mossad et al. [9] and the zinc acetate lozenge trial by Petrus et al. [10]. Both trials had close to 100 patients divided equally into their zinc and placebo groups (Table 1; Additional file 1).

The Mossad [9] trial

Figure 1 shows the number of common cold patients who recovered on each day of the follow-up in the Mossad [9] trial. The placebo group is on the top row and the zinc lozenge group on the bottom row of the figure. Both distributions appear closer to the uniform distribution than to the normal distribution. Furthermore, the SD of the placebo group is significantly greater than the SD of the zinc group (Table 1).

On the absolute scale, zinc lozenges shortened the duration of colds by 4.0 days. Table 1 shows the transformation of the common cold duration to the relative scale so that the placebo group is normalized to 100%. The zinc group level becomes the RoM = 0.566 and the difference between the zinc and placebo groups directly indicates a 43.4% reduction in common cold duration with the zinc lozenges. Since this transformation is linear, the t-tests for the 2 approaches lead to identical t-scores and P-values. The relevant question about the preference for the 2 effect estimates, 4.0 days vs. 43%, is the divergence in their implications.

If the 4-day reduction in common cold duration is assumed to be a uniform expected effect over Mossad’s patients then the 4-day estimate should be valid for both short and long colds. This implies that the 4-day reduction in cold duration with zinc lozenges corresponds to the translocation of the common cold distribution of the placebo group towards the left by 4.0 days (2nd row in Fig. 1). The uniform shortening of colds by 4.0 days predicts that zinc treatment would lead to 12 patients with a negative or zero cold duration, which are impossible. Such a transformation also retains the high variance of the placebo group compared with the actual zinc lozenge group.

If the 43% reduction in common cold duration is assumed to be a uniform relative effect over Mossad’s patients, this indicates that multiplication of the cold durations of the placebo group by 0.57 should lead to a distribution similar to the distribution of the zinc group (3rd and 4th rows in Fig. 1). The 43% shortening of colds in the placebo group does not predict impossible, i.e., negative or zero, cold durations and the SD of this predicted distribution is consistent with the SD of the actual zinc lozenge group (variance test P = 0.7).

The absolute and relative effects can also be compared on the basis of the quartiles of the cold distributions in the treatment groups (Table 1). The ratios for the 1st, 2nd and 3rd quartiles of the zinc and placebo groups indicate reductions in common cold duration by 40, 38 and 50% respectively with zinc lozenges, which are all close to the overall 43% estimate. In contrast, the absolute differences in the quartiles of the 2 groups are 2, 3 and 7 days, which have a much greater variation around the absolute effect estimate of 4.0 days. For the 90th percentiles of cold duration, the bootstrap 95% CI for the absolute difference between the zinc and placebo groups is from −11 to −5.6 days, which does not include the overall −4.0 day estimate. In contrast, the bootstrap of the ratio of the 90th percentiles gives a 95% CI from −58 to −31%, which is consistent with the overall estimate of −43%.

Survival analysis is a further informative way to examine the time of recovery. The Kaplan-Meier plots for the zinc group and the 43% shortened placebo group cold durations are closely overlapping (Fig. 2). Cox regression gives an estimate of RR = 2.7 for the effect of zinc lozenges.

The Petrus [10] trial

Figure 3 shows the number of patients recovering on each day in the Petrus [10] trial. Here too the distributions of colds in the placebo and zinc groups appear close to the uniform distribution and the SD is significantly greater in the placebo group than the zinc group (Table 1). Zinc lozenges shortened the duration of colds by 1.77 days on the absolute scale and by 25% on the relative scale (Table 1).

The 1.77-day uniform shortening of colds in the placebo group predicts that 3 colds will last just for 0 days (2nd row in Fig. 3) and retains the high variance compared with the actual zinc lozenge group. In contrast, the 25% shortening of colds in the placebo group leads to a cold distribution similar to the zinc lozenge group (3rd and 4th rows of Fig. 3), and to a SD consistent with that of the zinc group (variance test P = 0.4).

The quartiles of duration in the Petrus [10] trial are not informative for the comparison of the absolute and relative scales. However, the difference between the zinc and placebo groups at the 90th percentile level of duration, −43% and −6 days, is more consistent with the relative effect estimate of −25% compared with the absolute effect estimate of −1.77 days (Table 1). For the 90th percentiles, the bootstrap 95% CI for the difference between the groups is from −8 to −4.5 days, which is far from the overall estimate of −1.77 days. In contrast, the bootstrap of the ratio of the 90th percentiles gives a 95% CI from −53 to −33%, which is much closer to the overall estimate of −25%.

The Kaplan-Meier plots for the zinc group and the 25% shortened placebo group colds overlap closely (Fig. 4). Cox regression gives an estimate of RR = 1.77 for the rate of recovery in the zinc lozenge group compared with the placebo group.

If a treatment consistently has no effect at all, the scale of the meta-analysis probably matters little. However, if a treatment does have an effect, the scale may substantially influence the interpretation of the findings and the communication about them. Therefore this analysis selected 2 trials that found a significant benefit from zinc lozenges and for which IPD was available. While the goal of this study was not to estimate the overall effect of zinc lozenges on common cold duration, the findings in these 2 trials are consistent with the relative effects (i.e., the % decrease) found in 4 other trials on zinc lozenges [12, 19].

When the effect of zinc lozenges on cold duration was estimated on the absolute scale, i.e., as the number of days by which the colds became shorter, applying the effect estimates to the placebo group cold distribution yielded impossible, negative or zero, predicted durations for colds. In addition, the distributions remained wide compared with the distributions in the actual zinc lozenge groups. In contrast, when the relative effect estimate was applied to the placebo group cold distributions, no impossible common cold durations were predicted, and the cold distributions became similar to those of the zinc lozenge groups.

Calculation of the relative effect on common cold duration is conceptually consistent with the RR calculated by Cox regression. Nevertheless, the estimates should not be expected to be reciprocals since the RR depends on the shapes of the curves, which can differ substantially even though the mean durations remain invariant. Compared with the t-test, survival analysis is a superior way to analyse the recovery rate at the level of individual studies (Figs. 2 and 4), since it allows visual inspection and formal analysis of time-dependent changes in the treatment effect, and is not hampered by censored data or outlier patients who have exceptionally long colds. In contrast, outliers lead to inflation of the SD estimates in the t-test and may decrease the power to demonstrate an effect. However, since only the means and SDs of the study groups are usually published, survival analysis is rarely an option in the meta-analysis of disease duration. Therefore, in the meta-analyses of time data the usual question is about whether the absolute (i.e., MD) scale or the relative scale is more reasonable for the t-test.

The selection of scale is important in 2 respects. First, the scale is important when several studies are pooled in a meta-analysis, since one scale can lead to less heterogeneity than another, indicating better capture of the effect by the former scale [3]. Second, the scale influences the communication about the findings of single trials and of meta-analyses.

There has been substantial variation in the mean duration of untreated (placebo group) colds between trials. In our Cochrane review on vitamin C and the common cold, the shortest mean duration of placebo group colds in trials with children was 2.8 days, whereas the longest was 14 days [20, 21]. Because the untreated colds differ so greatly, a 1-day effect has a very different meaning in those two trials, even though the nominal value of the effect is the same. Furthermore, with an effective treatment, the 14-day colds might be shortened by a week, whereas such an effect does not exist on the 2.8-day scale of the former trial. Thus, the absolute scales of those two trials are incompatible. Therefore, we have used the relative scale in our Cochrane review on vitamin C and the common cold since 2004 [8, 20]. There has also been a substantial variation in cold durations in the placebo groups of 11 zinc lozenge studies, from 5 to 10 days [12].

There is evidence from studies on vitamin C and the common cold that heterogeneity in treatment effect may be lower on the relative scale. In 14 trials on vitamin C for children, the relative scale (i.e., the %-scale) pooling led to I² = 27% (P = 0.17) for the heterogeneity in the treatment effects, whereas heterogeneity in the days-scale was I² = 46% (P = 0.03). Less heterogeneity may lead to stronger evidence of treatment effect and, as expected, the pooled treatment effect on the relative scale was stronger (Z = 4.04) than on the absolute scale (Z = 3.11) [21]. An earlier meta-analysis of vitamin C and the common cold studies also pointed out that the relative scale led to a stronger evidence of treatment effect (P = 0.001) than the absolute scale (P = 0.01) [22]. These differences indicate that the relative scale may capture the effects of treatments on common cold duration better.

The selection of scale is also important in the communication of the findings. In the placebo groups of the included trials, the duration of the common colds ranged from 2 to 19 days in the Mossad [9] trial, and from 2 to 15 days in the Petrus [10] trial. Such a great variation in the durations of untreated colds should be taken into account in communication. The 43 and 25% effects are applicable over the whole range of the untreated cold durations. In contrast, claims that zinc shortens the duration of colds by 4.0 or 1.77 days according to the two trials analyzed (Table 1), or by 1.65 or 1.03 days according to two meta-analyses [13, 14], have a very different meaning depending on whether the assumed untreated cold episode might last for 2 days or 2 weeks. If only one type of estimate is used in the communication, the relative effect appears to be much more informative since it is applicable to the entire range of potential episode durations. Nevertheless, both measures may be shown in parallel.

The SMD scale is a third approach that has been used to estimate the magnitude of treatment effect on continuous outcomes [4, 5]. This normalizes the observations so that one unit on the scale corresponds to one unit of SD in each trial included in the meta-analysis. Such a scale is confusing for an ordinary reader. For example, the 2011 Cochrane review on zinc and the common cold stated in its abstract that the “intake of zinc is associated with a significant reduction in the duration (standardised mean difference (SMD) -0.97)” [23]. Reporting should always show the unit of measurement, and this sentence should have been written more accurately as: “zinc shortened the duration of colds by 0.97 SD units”. However, such accurate reporting would have revealed the main problem of the SMD scale: what does the SD unit mean in practical terms. Most physicians and patients can consider whether 42 or 25% is a small or a large effect, but few of them can form their own opinion about whether an effect of 0.97 SD units is small or large. In this respect, the relative scale is far superior in the communication of findings to physicians and patients, since they have long-term familiarity with the percentage effects. The difficulty of communicating the SMD findings to patients has been pointed out [5, 24].

The SMD scale is extensively used. However, its usage does not seem to originate from biological or statistical considerations, but from the fact that, along with the MD scale, it is the only option that is available in the popular statistical software for meta-analysis, such as the RevMan of Cochrane Collaboration [4]. Thus, the wide availability of the SMD option guides researchers to use it without them considering the biological issues properly, or the difficulties in communicating the findings on that scale [5, 24].

Based on 143 meta-analyses on continuous outcomes, Friedrich et al. concluded that there was less heterogeneity in the meta-analyses when they were carried out on the relative scale [5]. Although their findings support the preference for the relative scale, they did not consider the diversity of the kinds of outcomes that are measured by continuous outcomes. It seems evident that the relative scale is superior for some outcomes such as the duration of the common cold and other diseases, whereas the absolute scale may be superior for some other outcomes. Therefore diverse continuous outcomes should not be combined into a uniform mass, so that all of them are calculated on the relative scale and compared with all of them on the absolute scale.

Essential requirements for using the relative scale in a meta-analysis appear to be that the measurements can be transformed so that the control group means are 100%, and that there is a relevant and unambiguous target level of 0%. However, there are many types of continuous outcomes which do not have a relative scale that is relevant. For example, since blood pressure does not have either a reasonable 100% level over various trials, or a reasonable 0% target level, pooling studies and reporting effects on blood pressure on the percentage scale might lead to confusions. The mmHg scale (absolute scale) stratified by the pre-treatment mmHg level may be preferable. Another example where percentage scale would lead to confusions is measuring body temperature. If a patient has fever, it is more informative to describe how many degrees the fever was reduced, rather than describing the relative decrease in body temperature on the Kelvin scale.

The duration of the common cold was used as a model of continuous outcomes. The common cold is by itself a clinically relevant topic as reflected, for example, by the existence of 18 Cochrane reviews in which the title includes the term [25]. Furthermore, it seems evident that the findings of this study apply to many other continuous outcomes that measure time, such as the duration of other diseases, and the duration of hospital stay and intensive care unit stay, etc.

In this study, the transformation to the relative scale was done by dividing the mean and SD values by the placebo group mean value, which transforms the zinc group mean level to the RoM and keeps the ratios of SDs and means identical with their ratios on the absolute scale. This approach is transparent but more conservative than the Taylor series approach [5] and the Fieller approach [17], see Additional file 3. These approaches should be compared to find out which is the most useful in meta-analyses.

The choice between the absolute scale and the relative scale should be determined on the basis of biological reasoning and on the basis of testing which scale leads to less heterogeneity. The relative scale option should be made available for the standard meta-analysis software, but meanwhile the transformation to the relative scale can be easily calculated with spreadsheet programs, and the transformed data can be analyzed with standard meta-analysis software.