Bias-corrected estimator for intraclass correlation coefficient in the balanced one-way random effects model

  • Eshetu G Atenafu1Email author,

    Affiliated with

    • Jemila S Hamid2, 4Email author,

      Affiliated with

      • Teresa To3, 4, 5,

        Affiliated with

        • Andrew R Willan3, 4,

          Affiliated with

          • Brian M Feldman3, 4, 5 and

            Affiliated with

            • Joseph Beyene2, 3, 4, 5Email author

              Affiliated with

              BMC Medical Research Methodology201212:126

              DOI: 10.1186/1471-2288-12-126

              Received: 15 August 2011

              Accepted: 18 July 2012

              Published: 20 August 2012

              Abstract

              Background

              Intraclass correlation coefficients (ICCs) are used in a wide range of applications. However, most commonly used estimators for the ICC are known to be subject to bias.

              Methods

              Using second order Taylor series expansion, we propose a new bias-corrected estimator for one type of intraclass correlation coefficient, for the ICC that arises in the context of the balanced one-way random effects model. A simulation study is performed to assess the performance of the proposed estimator. Data have been generated under normal as well as non-normal scenarios.

              Results

              Our simulation results show that the new estimator has reduced bias compared to the least square estimator which is often referred to as the conventional or analytical estimator. The results also show marked bias reduction both in normal and non-normal data scenarios. In particular, our estimator outperforms the analytical estimator in a non-normal setting producing estimates that are very close to the true ICC values.

              Conclusions

              The proposed bias-corrected estimator for the ICC from a one-way random effects analysis of variance model appears to perform well in the scenarios we considered in this paper and can be used as a motivation to construct bias-corrected estimators for other types of ICCs that arise in more complex scenarios. It would also be interesting to investigate the bias-variance trade-off.

              Background

              The intraclass correlation coefficient (ICC), often denoted by ρ, was first introduced by Fisher [1] to study the familial resemblance between siblings. Since then it has obtained a wide range of applications in many areas such as psychology, epidemiology, genetics and genomics. See Donner [2] for an extensive review of inference procedures. In psychology, it plays a fundamental role in studying inter-rater reliability [3, 4]. It is used as a measure of heritability in classical genetic linkage studies to quantify the proportion of variance in traits of interest explained by genetic factors [5]. Intraclass correlation obtained from genome-wide association data has recently been used to provide a better estimate of heritability [6] . Sensitivity analysis is another application where ρ may be used as a means of investigating the effectiveness of an experimental treatment [7]. The intraclass correlation has also found some interesting application in genomics where it has been used to assess methodological and biological variations in DNA microarray analysis [8].

              The intraclass correlation coefficient also plays a key role in study design such as design of cluster randomized trials where it is traditionally used to quantify the degree of similarity between individuals within clusters [9, 10].

              Over the last decade, ICCs have received more attention in the literature and there has been an increasing awareness and appreciation of methodological issues related to these indices [1113].

              The most fundamental interpretation of ICCs is as a measure of the proportion of variance of a given outcome variable explained by a factor of interest in an analysis of variance model where it measures the relative homogeneity within groups [14, 15]. The first and essential step, therefore, is to specify an appropriate analysis of variance (ANOVA) model that best describes the study. The choice of the model is dictated by the specific situation defined by the experimental design and conceptual intent of the study [15]. Moreover, various forms of ICCs arise depending on the chosen model and the nature of the study [16, 17].

              For reasons mentioned above, inference procedures for ρ are closely related to the more general statistical problem of variance components [14, 18]. It is well known that estimation and hypothesis testing procedures for ICCs are, in general, sensitive to the assumption of normality and are subject to unstable variance [1, 19]. One, therefore, needs to consider normalizing and variance-stabilizing transformations on the basis of the rate of convergence to normality when constructing confidence intervals for the ICC. One of the well known and most commonly used normalization technique is Fisher’s Z transformation [1]. Other types of transformations have also been considered for the intraclass correlation coefficient [19, 20].

              Another important issue concerning ICCs is bias [21, 22]. The two most commonly used estimators, maximum likelihood and least square estimators, are known to be negatively biased. Although a Minimum Variance Unbiased (MVU) estimator for the intraclass correlation coefficient under two normal distributions is derived by [23], use of this estimator has been hindered because of absence of a closed form. Consequently, the MVU estimator is less widely recognized while the least square and maximum likelihood estimators are well-known. A computationally intensive FORTRAN subroutine is provided by Donoghue and Collins (1990).

              The purpose of this paper is, therefore, to provide a bias-corrected estimator for the intraclass correlation coefficient which is much simpler to compute and hence useful in practice. We consider a particular type of ICC in which we consider the estimation problem for ICC resulting from a one-way random effects analysis of variance model. We approximate the bias using a second-order Taylor series expansion and adjust the estimator to reduce the bias.

              The paper is organized as follows. We provide a brief background about the one-way random effects model and define the particular ICC of interest in Section “Methods”. In Section “Bias-corrected estimator for the intraclass correlation coefficient”, we propose a technique for approximating the bias resulting from the conventional estimator of ρ and we derive a new bias-corrected estimator for the parameter. We present simulation results in Section “Simulation Study” and provide a brief discussion in Section “Discussion”. Finally an Appendix consisting of some technical results is given at the end of the paper.

              Methods

              Consider n targets measured by k raters (instruments, judges etc.). A commonly used model for inferences concerning the intraclass correlation coefficient is the one-way random effects analysis of variance model where the j th target measurement by the i th rater (j = 1, 2, …, n ; i = 1, 2, …, k) can be described as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ1_HTML.gif
              (1)

              where it is assumed that http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq1_HTML.gif .

              The total sum of squares (Total SS) for the above model can be decomposed into two independent components as follows
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ2_HTML.gif
              (2)
              where SSE and SSB represent the within and between target sum of squares, respectively. The above decomposition is summarized in the analysis of variance table provided in Table 1. The table shows the source of variation, degrees of freedom (df), sums of squares (SS), mean square (MS), and expected mean square. We refer the reader to any standard analysis of variance text book [24] for details about this model.
              Table 1

              Analysis of variance table for one-way random effects model

              Source of Variation

              df

              SS

              MS=SS/df

              Expected MS

              Between Targets

              n-1

              SSB

              BMS

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq2_HTML.gif

              Within Targets

              n(k-1)

              SSE

              EMS

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq3_HTML.gif

              The intraclass correlation coefficient for the model in (1) is defined as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ3_HTML.gif
              (3)
              The most commonly used estimator for ρ, which is sometimes referred to as the analytical estimator, is given by
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ4_HTML.gif
              (4)

              Note that,

              • EMS is an unbiased estimator of http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq4_HTML.gif

              • (BMS-EMS)/k is an unbiased estimator of http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq5_HTML.gif

              Although the estimator in (4) is a ratio of unbiased estimators, it need not necessarily be unbiased itself. We consider the bias resulting from this estimator in the next section and provide a new bias-corrected estimator for the intraclass correlation coefficient.

              Bias-corrected estimator for the intraclass correlation coefficient

              Consider the intraclass correlation coefficient defined in (3) and re-write it as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ5_HTML.gif
              (5)

              An unbiased estimator for F is provided in the following theorem, which is useful in approximating the bias for estimating intraclass correlation coefficient. The variance of is also given in the theorem. The theorem has been considered by [25] in a different context.

              Theorem 1

              Consider http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq6_HTML.gif in equation (5), then
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equa_HTML.gif
              is an unbiased estimator for F. Moreover, its variance is given by
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equb_HTML.gif

              A proof of the theorem is provided in the Appendix.

              Let us now consider the following estimator for the intraclass correlation coefficient which is derived by substituting the unbiased estimator for F given in Theorem 1.
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ6_HTML.gif
              (6)

              As mentioned earlier, the estimator, http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq7_HTML.gif , in (6) need not be unbiased although it is a function of unbiased estimators. In fact, the bias is always negative and depends on the degree of correlation and the design size and balance [21].

              Now consider (5) and apply log transformation on both sides. The equation reduces to
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equc_HTML.gif
              An estimator for logρ, which is obtained by substituting F by its unbiased estimator, can be given as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equd_HTML.gif
              Note that the above estimator is equivalent to log http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq8_HTML.gif . The bias, in log scale can, therefore, be given as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Eque_HTML.gif
              Using a second order Taylor series approximation, we have
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equf_HTML.gif
              Consequently, the bias can be approximated as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ7_HTML.gif
              (7)
              It is important to note that the above approximation for the bias is always negative indicating that we are correcting the estimator from the right direction. A bias-corrected estimator for log ρ is obtained by adjusting for the bias given in (7) and is given by
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equg_HTML.gif
              where the subscript bc indicates that the estimator is bias corrected. http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq9_HTML.gif is an estimator for the http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq10_HTML.gif obtained by substituting F in http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq11_HTML.gif by http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq12_HTML.gif . We obtain a bias-corrected estimator for the intraclass correlation coefficient by transforming the above equation back to the original scale, and it is given by
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equ8_HTML.gif
              (8)
              In situations where F is small, we consider Taylor expansion of 1−ρ instead. Consequently,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equh_HTML.gif

              It can be shown using our approximation that the bias in general decreases as the degree of correlation moves away from 0.5. That is, the bias is small for both weak and strong correlations. Our simulation results confirm that the bias resulting from the analytical estimator is indeed small when the true value of ρis small (weak correlation) or large (strong correlation) (see Section “Discussion” for more details). As a result, we expect the performance of the conventional estimator to improve in such cases. In fact, previous simulation results have also showed that the analytical estimator, http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq13_HTML.gif , performs well for small values of ρ both for normal and non-normal data (e.g., producing confidence intervals close to the nominal level [10]. Our simulation results show that the estimators we proposed in this paper provide a considerable bias reduction even under such circumstances (see Section “Discussion” for details).

              Simulation Study

              We carried out extensive simulations to evaluate the performance of our bias-corrected estimator http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq14_HTML.gif . The bias resulting from our estimator is compared with bias from the conventional (analytical) estimator using normal as well as non-normal data. It is to be recalled that we based the Taylor expansion around http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq15_HTML.gif which a variant of the conventional estimator. We have, therefore, provided the bias resulting from this estimator for comparison purposes.

              Simulation Design

              Data were simulated as in [10], with slight modifications to the number of configurations that were allowed to vary. We used 3x8x2 design instead of their 3x4x2 configuration. That is, we considered 3 cluster sizes (10,30,50); 8 intraclass correlation coefficients (0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9) instead of only 4 true ICC values used in [10]; and two types of data distributions (normal versus non-normal), for a total of 48 simulation configurations.

              For normal outcome, data were simulated according to the framework of the one-way random effects model described in Section “Methods”. They were generated as the sum of two independent random variables http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq16_HTML.gif and http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq17_HTML.gif . When simulating non-normal outcome, the e ij alone were generated from a normal distribution with the a i generated from a Gamma distribution with shape parameter α = 1.67 and scale parameter http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq18_HTML.gif so that skewness of the distribution will be http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq19_HTML.gif and a kurtosis coefficient of http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq20_HTML.gif . The skewness and kurtosis coefficients represent marked deviation from normality. The skewness and kurtosis coefficients for a normally distributed random variable are 0 and 3, respectively.

              Without loss of generality, the overall mean was fixed at 10. Moreover, the sum of the variance components was constrained to be 1000, for both normal and non-normal data sets. The two variances, http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq21_HTML.gif and http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq22_HTML.gif , were then systematically manipulated to generate data from a population with the required ICC value using the following relationship
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equi_HTML.gif

              We simulated 5000 replications of data for each of the 48 scenarios. Different seeds were used for the random number generator at each replication while keeping it the same across different methods.

              Results

              Estimates of ICC along with percentage of bias for non-normal and normal data, averaged over 5000 simulations, are summarized in Table 2 and Table 3, respectively. As expected, the bias resulting from the analytical estimator is negative both for normal and non-normal data sets. Moreover, the bias gets smaller as the true value of ρ moves further away from 0.5 (see Figure 1). Furthermore, the biases resulting from the conventional estimator ( http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq23_HTML.gif ) and its variant ( http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq24_HTML.gif ) for the non-normal sample are larger relative to the normal sample indicating that the estimator is sensitive to the normality assumption. The difference is much larger for small cluster sizes. For cluster size 10 from the non-normal data with moderate correlation (ρ = 0.4,0.5,0.6), for example, the analytical estimator and its variant gave about 12% and 13% bias, respectively. However, the corresponding biases for normal data with the same cluster size and similar ρ values are 6% and 7%, respectively. It is, therefore, not recommended to use the analytical estimator for non-normal data, especially when there is moderate correlation. It is also important to note that the bias resulting from http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq25_HTML.gif is uniformly smaller than that of http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq26_HTML.gif where the difference is much larger for small cluster sizes.
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Fig1_HTML.jpg
              Figure 1

              Plot of the percentage bias against the true value of ICC for the normal (left panel) and non-normal (right panel) data sets, where solid line and dashed line represent the analytical and bias-corrected estimators, respectively.

              Table 2

              Simulation results for the non-normal data

                

              Estimate

              % Bias

              Clusters

              ICC

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq27_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq28_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq29_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq30_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq31_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq32_HTML.gif

              10

              0.1

              0.0927

              0.0891

              0.0941

              -7.3

              -11.0

              -5.9

               

              0.2

              0.1813

              0.1768

              0.1911

              -9.4

              -12.0

              -4.5

               

              0.3

              0.2678

              0.2627

              0.2890

              -11.0

              -12.0

              -3.7

               

              0.4

              0.3543

              0.3488

              0.3865

              -11.0

              -13.0

              -3.4

               

              0.5

              0.4423

              0.4367

              0.4830

              -12.0

              -13.0

              -3.4

               

              0.6

              0.5337

              0.5282

              0.5785

              -11.0

              -12.0

              -3.6

               

              0.7

              0.6306

              0.6256

              0.6744

              -9.9

              -11.0

              -3.7

               

              0.8

              0.7360

              0.7319

              0.7743

              -8.0

              -8.5

              -3.2

               

              0.9

              0.8552

              0.8526

              0.8809

              -5.0

              -5.3

              -2.1

              30

              0.1

              0.0977

              0.0964

              0.0977

              -2.3

              -3.6

              -2.3

               

              0.2

              0.1935

              0.1919

              0.1954

              -3.3

              -4.1

              -2.3

               

              0.3

              0.2883

              0.2865

              0.2954

              -3.9

              -4.5

              -1.5

               

              0.4

              0.3830

              0.3810

              0.3951

              -4.3

              -4.8

              -1.2

               

              0.5

              0.4783

              0.4763

              0.4929

              -4.3

              -4.7

              -1.4

               

              0.6

              0.5751

              0.5733

              0.5899

              -4.2

              -4.4

              -1.7

               

              0.7

              0.6745

              0.6728

              0.6880

              -3.6

              -3.9

              -1.7

               

              0.8

              0.7773

              0.7760

              0.7883

              -2.8

              -3 .0

              -1.5

               

              0.9

              0.8851

              0.8844

              0.8919

              -1.7

              -1.7

              -0.9

              50

              0.1

              0.0984

              0.0977

              0.0984

              -1.6

              -2.3

              -1.6

               

              0.2

              0.1957

              0.1947

              0.1965

              -2.1

              -2.6

              -1.8

               

              0.3

              0.2924

              0.2912

              0.2966

              -2.5

              -2.9

              -1.1

               

              0.4

              0.3890

              0.3878

              0.3968

              -2.8

              -3.1

              -0.8

               

              0.5

              0.4862

              0.4850

              0.4951

              -2.8

              -3.0

              -1.0

               

              0.6

              0.5844

              0.5832

              0.5931

              -2.6

              -2.8

              -1.2

               

              0.7

              0.6842

              0.6832

              0.6921

              -2.3

              -2.4

              -1.1

               

              0.8

              0.7861

              0.7854

              0.7925

              -1.7

              -1.8

              -0.9

               

              0.9

              0.8911

              0.8907

              0.8949

              -1.0

              -1.0

              -0.6

              Table 3

              Simulation results for the normal data

                

              Estimate

              % Bias

              Clusters

              ICC

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq33_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq34_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq35_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq36_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq37_HTML.gif

              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq38_HTML.gif

              10

              0.1

              0.0969

              0.0932

              0.0979

              -3.2

              -6.8

              -2.1

               

              0.2

              0.1905

              0.1858

              0.2001

              -4.8

              -7.1

              0.1

               

              0.3

              0.2832

              0.2778

              0.3071

              -5.6

              -7.4

              2.4

               

              0.4

              0.3759

              0.3701

              0.4140

              -6.0

              -7.5

              3.5

               

              0.5

              0.4696

              0.4637

              0.5158

              -6.1

              -7.3

              3.2

               

              0.6

              0.5652

              0.5596

              0.6135

              -5.8

              -6.7

              2.3

               

              0.7

              0.6641

              0.6591

              0.7094

              -5.1

              -5.8

              1.3

               

              0.8

              0.7677

              0.7638

              0.8049

              -4.0

              -4.5

              0.6

               

              0.9

              0.8784

              0.8760

              0.9016

              -2.4

              -2.7

              0.2

              30

              0.1

              0.0995

              0.0983

              0.0995

              -0.5

              -1.7

              -0.5

               

              0.2

              0.1976

              0.1960

              0.1989

              -1.2

              -2.0

              -0.6

               

              0.3

              0.2953

              0.2934

              0.3029

              -1.6

              -2.2

              1.0

               

              0.4

              0.3929

              0.3909

              0.4067

              -1.8

              -2.3

              1.7

               

              0.5

              0.4910

              0.4889

              0.5063

              -1.8

              -2.2

              1.3

               

              0.6

              0.5897

              0.5878

              0.6047

              -1.7

              -2.0

              0.8

               

              0.7

              0.6895

              0.6878

              0.7030

              -1.5

              -1.7

              0.4

               

              0.8

              0.7908

              0.7895

              0.8015

              -1.2

              -1.3

              0.2

               

              0.9

              0.8941

              0.8934

              0.9005

              -0.7

              -0.7

              0.1

              50

              0.1

              0.0990

              0.0983

              0.0990

              -1.0

              -1.7

              -1.0

               

              0.2

              0.1978

              0.1969

              0.1984

              -1.1

              -1.6

              -0.8

               

              0.3

              0.2965

              0.2954

              0.3009

              -1.2

              -1.5

              0.3

               

              0.4

              0.3952

              0.3940

              0.4037

              -1.2

              -1.5

              0.9

               

              0.5

              0.4942

              0.4929

              0.5033

              -1.2

              -1.4

              0.7

               

              0.6

              0.5935

              0.5924

              0.6024

              -1.1

              -1.3

              0.4

               

              0.7

              0.6936

              0.6926

              0.7015

              -0.9

              -1.1

              0.2

               

              0.8

              0.7945

              0.7938

              0.8008

              -0.7

              -0.8

              0.1

               

              0.9

              0.8966

              0.8961

              0.9002

              -0.4

              -0.4

              0.0

              In general, a considerable bias reduction has been obtained by using our bias-corrected estimator. This is true for all values of ρand all cluster sizes although the improvement is much larger for moderate correlations (see Figure 1). Moreover, the improvement obtained for the non-normal sample is relatively larger than that obtained for the normal sample. For the non-normal sample, for instance, using the conventional estimator resulted in 12% bias whereas only 3.4% bias was obtained from our estimator for cluster size 10 and moderate correlations. For the normal sample and the same scenario, biases resulting from the conventional estimator and the bias-corrected estimator are 6% and 3%, respectively. Improvements are also obtained for small or large ρ values, that is, in situations where the bias from the conventional estimator is small. For instance, for cluster size 10 from the non-normal sample with ρ = 0.2, 9.4% and 12% biases were obtained using http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq39_HTML.gif and http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq40_HTML.gif , respectively, whereas only 4.5% bias was observed for our estimator. For the normal sample with the same cluster size and correlation, http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq41_HTML.gif and http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq42_HTML.gif resulted in 4.8% and 7.1% biases. In this situation, the bias reduced to 0.05% when using our bias corrected estimator. Similar statements can be made for large ICC values.

              Discussion

              The intraclass correlation coefficient (ICC) has widespread applications from measuring heritability in genetic studies to measuring reliability, consistency and agreement of measurements in a host of clinical, biomedical and psychosocial areas. The ICC has important role in study design and sample size calculations as well. For instance, designs of family-based genetic studies can be greatly impacted by the estimated ICC (often referred to as the coefficient of heritability in the genetics literature). Similarly, trials involving clustering of some degree (e.g., longitudinal study design, multilevel models, cluster randomized trials, etc.) will be influenced to various extent by the magnitude of the intraclass correlation coefficient. Because ICC estimates have great implications to design considerations, statistical analysis as well as interpretation of study findings, it is critical to use an estimator with minimal bias.

              In this paper, we proposed a new bias-corrected estimator for one type of intraclass correlation coefficient. We used a variant of the conventional estimator (ANOVA estimator) and applied Taylor series expansion to approximate the bias. The approximate bias was then used and a new adjusted estimator is proposed. The bias-corrected estimator proposed in this paper is much simpler to compute than the minimum variance unbiased (MVU) estimator of Olkin and Pratt [23]. Moreover, our simulation study shows that our estimator outperforms the conventional estimator by providing a substantial decrease in the bias. For small cluster sizes from normal data, however, a positive bias was introduced although the percentage bias resulted from our estimator is still smaller than that of the ANOVA estimator. This might be improved by using second order Taylor series expansion instead of using only the first order adopted in this paper.

              Conclusion

              We considered a particular type of intraclass correlation coefficient that arises from a one-way random effects analysis of variance model, although the method can be extended to provide bias-corrected estimators for other types of ICCs. Furthermore, the current paper is focused on bias reduction in a balanced data setting, and we plan to investigate other optimality measures as well as the performance of the bias-corrected method for unbalanced data when the number of observations differ from cluster to cluster. Finally, we would like to highlight that ICCs are subject to different interpretations, so the user should apply the various ICCs with caution [17, 2628].

              Appendix

              Recall from the one-way ANOVA model http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq43_HTML.gif . Therefore, one can easily show that http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq44_HTML.gif and http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq45_HTML.gif Moreover, http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq46_HTML.gif . Consequently,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equj_HTML.gif

              Which implies that http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq47_HTML.gif .

              Moreover,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equk_HTML.gif
              As a result we have,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equl_HTML.gif
              Now applying results from [24], we get
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equm_HTML.gif
              After a simple algebraic manipulations, we can show that
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equn_HTML.gif

              which proves the first part of the theorem.

              Now consider,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equo_HTML.gif
              Consequently,
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equp_HTML.gif
              Note that http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_IEq48_HTML.gif The variance is given as
              http://static-content.springer.com/image/art%3A10.1186%2F1471-2288-12-126/MediaObjects/12874_2011_863_Equq_HTML.gif

              Declarations

              Acknowledgements

              We would like to gratefully acknowledge a seed grant support provided by the Research Institute of the Hospital for Sick Children, Toronto, Ontario, Canada.

              Authors’ Affiliations

              (1)
              Princess Margaret Hospital
              (2)
              Department of Clinical Epidemiology and Biostatistics, McMaster University
              (3)
              Child Health Evaluative Sciences Hospital for Sick Children
              (4)
              Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto
              (5)
              Departments of Pediatrics, Health Policy and Management, University of Toronto

              References

              1. Fisher RA: On the “probable error” of a coefficient of correlation deduced from a small sample. Metron 1921, 1:1–32.
              2. Donner A: A review of inference procedures for the intraclass correlation coefficient in the one-way random effects model. Int Stat Rev 1986, 54:67–82.View Article
              3. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N: The dependability of behavioral measurements: theory of generalizability od scores and profiles. New York: Wiley; 1972.
              4. Ebel RL: Estimation of the reliability of ratings. Psychometrika 1951, 16:407–424.View Article
              5. Visscher PM: On the sampling variance of intraclass correlations and genetic correlation. Genetics 1998, 149:1605–1614.PubMed
              6. Visscher PM, et al.: Assumption-free estimation of heritability from genome-wide identity-by-descent sharing between full sibilings. PLoS Genet 2006,2(3):e41.PubMedView Article
              7. Bradley RA, Schumann DEW: The comparison of the sensitivities of similar experiments: applications. Biometrics 1957, 13:496.View Article
              8. Pellis L, Hal NL, von Burema J, Keijer J: The intraclass correlation coefficient applied for evaluation of data correction, labeling methods, and rectal biopsy sampling in DNA microarray experiments. Physiol Genomics 2003, 16:99–106.PubMedView Article
              9. Killip S, Mahfoud Z, Pearce K: What is an intracluster correlation coefficient? Crucial concepts for primary care researchers: crucial concepts for primary care researchers. Ann Family Med 2003,2(3):204–208.View Article
              10. Non parametric bootstrap confidence intervals for the correlation coefficient Stat Med 2003, 22:3805–3821.
              11. Altaye M, Donner A, Klar N: Inference procedures for assessing interobserver agreement among multiple raters. Biometrics 2001, 457:584–588.View Article
              12. Ghosh H, Das A: Optimal diallel cross designs for estimation of heritability. J Stat Planning and Inference 2003,116(1):185–196.View Article
              13. Shoukri MM, Donner A: Efficiency considerations in the analysis of inter-observer agreement. Biostatistics 2001, 2:321–336.View Article
              14. Fisher RA: Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd; 1925.
              15. Shrout PE, Fleiss JL: Intraclass correlations: uses in assessing rater reliability. Psychological Bull 1979, 86:420–428.View Article
              16. Fleiss JL, Cohen J: The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Meas 1973, 33:613–619.View Article
              17. Müller R, Büttner P: A critical discussion of intraclass correlation coefficients. Stat Med 1994, 13:2465–2476.PubMedView Article
              18. Snedecor GW: Statistical Methods. Ames: The Iowa State University Press; 1946.
              19. Konishi S: Normalizing and variance stabilizing transformations for intraclass correlation. Ann Inst Statist Math 1985, 37:Part A 87–94.View Article
              20. Weinberg R, Patel YC: Simulated intraclass correlation coefficients and their z transforms. J Stat Comput Simul 1981, 13:13–26.View Article
              21. Wang CS, Yadell BS, Rutledge JJ: Bias of maximum likelihood estimator of intraclass correlation. Theor Appl Genet 1991, 82:421–424.View Article
              22. Ponzoni RW, James JW: Possible biases in heritability estimates from intraclass correlation. Theor Appl Genet 1978, 53:25–27.
              23. Olkin I, Pratt W: Unbiased estimation of certain correlation coefficients. Ann Mathamatical Stat 1958, 29:201–211.View Article
              24. Snedecor GW, Cochran WG: Statistical Methods. Ames: The Iowa State University Press; 1980.
              25. Das A, Ghosh H: Estimation of the ratio of variance components and optimal diallel cross over designs. Indian Statistical Institute 2003. Technical Report
              26. Barnhart HX, Haber MJ, Lin LI: An overview on assessing agreement with continuous measurements. J Biopharmaceutical Stat 2007, 17:529–569.View Article
              27. Chen CC, Barnhat HX: Comparison of ICC and CCC for assessing agreement for data without and with replications. Comput Stat Data Anal 2008, 53:554–564.View Article
              28. Costa-Santos C, Bernardes J, Ayres-De-Campos D, Costa A, Costa C: The limits of agreement and the intraclass correlation coefficient may be inconsistent in the interpretation of agreement. J Clin Epidemiol 2011, 64:264–269.PubMedView Article
              29. Pre-publication history

                1. The pre-publication history for this paper can be accessed here:http://​www.​biomedcentral.​com/​1471-2288/​12/​126/​prepub

              Copyright

              © Atenafu et al.; licensee BioMed Central Ltd. 2012

              This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://​creativecommons.​org/​licenses/​by/​2.​0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.