We use Mathematica software, version 8, to calculate the expectation and bias of *I*^{2} analytically. This Methods section introduces notation, assumptions, and statistical properties, and describes the calculations that we submitted to Mathematica. The Results section will give the results of those calculations.

### Meta-analysis

Meta-analysis summarizes the results of *K* studies, each of which has sample size *n*_{
k
}, *k* = 1,…,*K*. In each study, there is a true effect *β*_{
k
} estimated by \( {\widehat{\beta}}_k \), with a true standard error *σ*_{
k
} estimated by \( {\widehat{\sigma}}_k \), or, equivalently, a true variance \( {\sigma}_k^2 \) estimated by \( {\widehat{\sigma}}_k^2 \). With large *n*_{
k
}, the quantity \( \left({\widehat{\beta}}_k-{\beta}_k\right)/{\widehat{\sigma}}_k \) approaches a standard normal distribution according to the central limit theorem.

Two models can be used in meta-analysis: a *fixed-effects* model and a *random-effects* model. Some confusion is possible because the term fixed effects is used in two different senses [15]. In some literature, the term fixed effects means that the *K* study effects *β*_{
k
} are assumed to be homogeneous. We use the term fixed effects in its other sense, where it means that we seek only to generalize about the *K* studies in the meta-analysis. The true effects *β*_{
k
} can be either homogeneous or heterogeneous, but they are regarded as fixed quantities. Because of sampling error, the *K* studies would produce different estimates \( {\widehat{\beta}}_k \) and \( {\widehat{\sigma}}_k \) if they were repeated, but the true effects *β*_{
k
} and true standard errors *σ*_{
k
} would not change.

Under a random-effects model, by contrast, we assume that the true effects *β*_{
k
} in the meta-analysis were drawn at random from a larger population of effects, and we seek to make inferences about that larger population [16]. So the *β*_{
k
} are not fixed quantities but random variables that would be different if a different sample were drawn from the population of effects.

### The estimand *ι*^{2}

In order to understand the properties of the estimator *I*^{2}, we must first define the quantity that is being estimated. We call the estimand *ι*^{2}. It represents the fraction of variance in the estimated effects \( {\widehat{\beta}}_k \) that is due to heterogeneity rather than measurement error.

More formally, the \( {\widehat{\beta}}_k \) vary from one study to another. The variance in \( {\widehat{\beta}}_k \) is partly due to the heterogeneity of the true effects *β*_{
k
} and partly due to estimation error summarized by the standard errors *σ*_{
k
}. By the law of total variance we have

$$ \begin{array}{c}V\left({\widehat{\beta}}_k\right)=V\left({\beta}_k\right)+E\left({\sigma}_k^2\right)\\ {}={\tau}^2+{\sigma}^2\end{array} $$

(1)

where *τ*^{2} = *V*(*β*_{
k
}) is the heterogeneity variance or between-study variance, and \( {\sigma}^2=E\left({\sigma}_k^2\right) \) is the average within-study variance. Under a fixed-effects model these variances and expectations refer only to the *K* effects *β*_{
k
} and standard errors *σ*_{
k
} in the meta-analysis. Under a random effects model *τ*^{2} refers to the larger population of effects, but *σ*^{2} still refers only to the *K* standard errors *σ*_{
k
} in the meta-analysis, unless we are willing to regard the *σ*_{
k
} as well as the *β*_{
k
} as samples from a larger population.

The fraction of variance that is due to heterogeneity is

$$ {\iota}^2=\frac{V\left({\beta}_k\right)}{V\left({\widehat{\beta}}_k\right)}=\frac{\tau^2}{\tau^2+{\sigma}^2} $$

(2)

If *ι*^{2} = 0 then the effects *β*_{
k
} are homogeneous; if *ι*^{2} > 0 then they are heterogeneous.

Note that, unlike some past definitions [6], our definition of *ι*^{2} does not assume equal standard errors *σ*_{1} = *σ*_{2} = … = *σ*_{
K
}. Note also that *ι*^{2} is not an absolute measure of heterogeneity. Instead, *τ*^{2} is an absolute measure of heterogeneity, while *ι*^{2} compares *τ*^{2} to *σ*^{2}. When the estimation error is small, as it is if *n*_{
k
} is large, then *ι*^{2} can be large even if *τ*^{2} is small [17].

### The naïve estimator \( {\widehat{\boldsymbol{\iota}}}^{\mathbf{2}} \)

To estimate the fraction *ι*^{2}, Higgins and Thompson [6] first derived the naïve estimator

$$ {\widehat{\iota}}^2=1-\frac{df}{Q} $$

(3)

where *df* = *K*–1, *Q* is Cochran’s *Q* statistic [4]

$$ Q={\displaystyle \sum_{k=1}^K}\frac{{\left({\widehat{\beta}}_k-\widehat{\overline{\beta}}\right)}^2}{{\widehat{\sigma}}_k^2} $$

(4)

and

$$ \widehat{\overline{\beta}}=\frac{{\displaystyle {\sum}_{k=1}^K}{\widehat{\sigma}}_k^{-2}{\widehat{\beta}}_k}{{\displaystyle {\sum}_{k=1}^K}{\widehat{\sigma}}_k^{-2}} $$

(5)

is the precision-weighted average of the estimated effects.

The distribution of \( {\widehat{\iota}}^2 \) depends on the distribution of *Q*. Under homogeneity, with large *n*_{
k
}, *Q* has a central chi-square distribution with *df* degrees of freedom.

Under heterogeneity, the large-*n*_{
k
} distribution of *Q* depends on whether we regard the effects as fixed or random. Under a random-effects model, *Q* is distributed like a weighted sum of *K*–1 central \( {\chi}_1^2 \) variables, where the weights are given by a matrix function of *τ*^{2} and \( {\sigma}_k^2 \) [18]. If we make the simplifying assumption that all the standard errors are equal (*σ*_{
k
}*= σ*) then the weights are all equal to 1 + *τ*^{2}/*σ*^{2} [18] or, in our notation (1 − *ι*^{2})^{− 1}, so that

$$ X=\left(1-{\iota}^2\right)Q $$

(6)

has a central chi-square distribution with *df* degrees of freedom [18]. As *ι*^{2} gets small, we converge toward the homogeneous situation where *Q* itself has a central chi-square distribution with *df* degrees of freedom.

Under a fixed-effects model, by contrast, *Q* has a non-central chi-square distribution with *df* degrees of freedom and a non-centrality parameter of [19]

$$ \lambda ={\displaystyle \sum_{k=1}^K}\frac{{\left({\beta}_k-\overline{\beta}\right)}^2}{\sigma_k^2} $$

(7)

where \( \overline{\beta} \) is the precision-weighted mean of the true effects *β*_{
k
}. If we make the simplifying assumption that all the standard errors are equal (*σ*_{
k
}*= σ*) then the non-centrality parameter reduces to

$$ \begin{array}{c}\lambda =\frac{1}{\sigma^2}{\displaystyle \sum_{k=1}^K}{\left({\beta}_k-\overline{\beta}\right)}^2\\ {}=K\frac{\tau^2}{\sigma^2}\\ {}=K\frac{\iota^2}{1-{\iota}^2}\end{array} $$

(8)

The last line shows that *λ* is an increasing function of *ι*^{2} and that that *λ* = 0 if *ι*^{2} = 0. So again, as *ι*^{2} gets small, *Q* converges toward the central chi-square distribution that it has under homogeneity.

### The truncated estimator *I*^{2}

A shortcoming of the naïve estimator \( {\widehat{\iota}}^2 \) is that it can be negative even though the estimand *ι*^{2} cannot. Negative values of \( {\widehat{\iota}}^2 \) occur whenever *Q* < *df*, which is not a rare event. Figure 1 shows the probability that \( {\widehat{\iota}}^2 \) is negative when the effects are homogeneous. The probability decreases as *df* increases, but the probability is always greater than 50%.

To avoid negative estimates, Higgins and Thompson [6] suggested rounding them up to zero. The rounded or truncated estimator

$$ {I}^2= \max \left(0,{\widehat{\iota}}^2\right) $$

(9)

is the estimator that is widely used today. *I*^{2} cannot be negative but can be zero. Values of *I*^{2} = 0 occur in about one-quarter of published meta-analyses [20].

### Expectation and bias of the estimators

The expectation of the naïve estimator \( {\widehat{\iota}}^2 \) is

$$ E\left({\widehat{\iota}}^2\right)=1-df\ E\left(\frac{1}{Q}\right) $$

(10)

This is easily calculated in the homogeneous case, where 1/*Q* is an inverse chi-square variable whose expectation is 1/(*df* – 2). It is just as easily calculated in the heterogeneous case with fixed effects; in that case, 1/*Q* is a scaled inverse chi-square variable with an expectation of (1 − *ι*^{2})/(*df* − 2). The calculation is harder in the heterogeneous case with random effects; in that case, 1/*Q* is the scaled inverse of a noncentral chi-square variable. Although the expectation of this inverse has a closed-form solution [21], it is not transparent or easy to calculate by hand. However, we can calculate it using Mathematica.

The expectation of the truncated estimator *I*^{2} is a little harder to calculate. It is the weighted average of two conditional expectations: the expectation of *I*^{2} when *I*^{2} = 0 and the expectation of *I*^{2} when *I*^{2} > 0. The probability that *I*^{2} = 0 is *P*(*Q* < *df*), and the probability that *I*^{2} > 0 is *P*(*Q* > *df*). Therefore the expectation of *I*^{2} is

$$ \begin{array}{c}E\left({I}^2\right)=P\left(Q<df\right)\times 0+P\left(Q>df\right)\times E\left({I}^2\left|Q>df\right.\right)\\ {} = P\left(Q>df\right)\times E\left(1-\frac{df}{Q}\left|Q>df\right.\right)\end{array} $$

(11)

Under homogeneity, *Q* has a central chi-square distribution and the expectation *E*(*I*^{2}) has a closed-form solution which Mathematica can calculate.

Under heterogeneity, the expectation *E*(*I*^{2}) depends on whether we regard the effects as fixed or random. If effects are random, then *X* = (1 − *ι*^{2})*Q* has a central chi-square distribution. The probability that *I*^{2} = 0 is *P*(*X* < (1 − *ι*^{2})*df*), and the probability that *I*^{2} > 0 is *P*(*X* > (1 − *ι*^{2})*df*). Therefore the expectation of *I*^{2} is

$$ E\left({I}^2\right) = P\left(X>\left(1-{\iota}^2\right)df\right)\times E\left(1-\left(1-{\iota}^2\right)\frac{df}{X}\left|X>\left(1-{\iota}^2\right)df\right.\right) $$

(12)

which again has a closed-form solution which Mathematica can calculate.

If instead effects are fixed, then the expectation *E*(*I*^{2}) in (11) has no closed-form solution. But the expectation for specific values of *ι*^{2} and *df* can be calculated using numerical integration in Mathematica.