Analysis of human immune responses in quasiexperimental settings: tutorial in biostatistics
 Rajiv Sarkar^{1},
 Sitara S Ajjampur^{1},
 Honorine D Ward^{1, 2},
 Gagandeep Kang^{1} and
 Elena N Naumova^{1, 3}Email author
DOI: 10.1186/14712288121
© Sarkar et al; licensee BioMed Central Ltd. 2012
Received: 28 November 2010
Accepted: 3 January 2012
Published: 3 January 2012
Abstract
Background
Human immunology is a growing field of research in which experimental, clinical, and analytical methods of many life science disciplines are utilized. Classic epidemiological study designs, including observational longitudinal birth cohort studies, offer strong potential for gaining new knowledge and insights into immune response to pathogens in humans. However, rigorous discussion of methodological issues related to designs and statistical analysis that are appropriate for longitudinal studies is lacking.
Methods
In this communication we address key questions of quality and validity of traditional and recently developed statistical tools applied to measures of immune responses. For this purpose we use data on humoral immune response (IR) associated with the first cryptosporidial diarrhea in a birth cohort of children residing in an urban slum in south India. The main objective is to detect the difference and derive inferences for a change in IR measured at two time points, before (pre) and after (post) an event of interest. We illustrate the use and interpretation of analytical and data visualization techniques including generalized linear and additive models, datadriven smoothing, and combinations of box, scatter, and needleplots.
Results
We provide stepbystep instructions for conducting a thorough and relatively simple analytical investigation, describe the challenges and pitfalls, and offer practical solutions for comprehensive examination of data. We illustrate how the assumption of time irrelevance can be handled in a study with a prepost design. We demonstrate how one can study the dynamics of IR in humans by considering the timing of response following an event of interest and seasonal fluctuation of exposure by proper alignment of time of measurements. This alignment of calendar time of measurements and a child's age at the event of interest allows us to explore interactions between IR, seasonal exposures and age at first infection.
Conclusions
The use of traditional statistical techniques to analyze immunological data derived from observational human studies can result in loss of important information. Detailed analysis using welltailored techniques allows the depiction of new features of immune response to a pathogen in longitudinal studies in humans. The proposed staged approach has prominent implications for future study designs and analyses.
Background
Human immunology is a growing field and includes methodologies of many experimental and clinical disciplines: molecular biology, microbiology, immunogenetics, clinical immunology, pathophysiology, epidemiology, and potentially others. The essence of scientific proof in human immunology employs a set of applicable and ethically acceptable rules. The direct interpolation of techniques developed for fully controlled experimental designs can be a challenging task. The profound differences in clinical, epidemiological, and laboratory studies have to do with basic assumptions, which logically define a research hypothesis and analytical procedures we apply to test this hypothesis. For example, in a study aimed to examine the effect of "an event", say "infection by a pathogen" on a marker of an immune response such as antibody levels, a design or protocol for measuring such an effect in a fully controlled experiment may differ dramatically in a murine model and in a cohort of newborn children. It is important to know if the measurements that are used to judge the effect were obtained from the same subjects or not, because this aspect of a study design will affect the choice of the statistical test. If a study subject contributes two measurements: one before "an event of interest", also called a baseline measure and another measurement taken after an event, we are dealing with a socalled "repeated measurement" scenario, a prepost design, which is the focus of this communication.
Reviewing the recent literature one can easily notice that a typical statistical analysis conducted in a traditional prepost design is often limited to a paired ttest. In some instances this analytical scheme is sufficient. However, in many situations it can be overly simplistic since it does not take into account the complexity of data collection protocols and various issues related to field research, ignores underlying theoretical assumptions that are essential for proper use of statistical tests, and discounts the important confounding factors associated with immune responses in humans. In this communication we address key questions of quality and validity of a statistical analysis performed by measuring human immune responses in a longitudinal setting. By using an example, we provide stepbystep instructions, and describe the pitfalls and solutions in a comprehensive examination of data.
The organization of the manuscript is as follows. First, we will introduce an example borrowed from an actual study to detail the study design, experimental procedures, and notations. Next, we present the traditional statistical approach to a prepost comparison of study outcomes in controlled experimental conditions, and describe our analytical plan in several steps with an emphasis on developing supportive visual aids. We then present the rationale for further explorations into the nature of prepost changes and suggest an approach to such examinations by investigating the effect of an "outlier" as an example. We detail the potential flaws of the traditional analytic approach applied to longitudinal settings of measuring human immune responses in "realworld" conditions. We focus on the important consideration of the measurements' timing, age of the subject, and seasonal effects. We demonstrate how novel analytical techniques can be applied to reveal unique features of immune response dynamics, agespecific elevation in the immune response, and the effect of seasonal synchronization that explains a large fraction of variability of temporal changes. We conclude this tutorial with recommendations for the use of this proposed scheme in other practical settings.
Methods
Motivating Example
To illustrate the analytical process we used data collected from a birth cohort of children observed over three years in a semiurban community in south India. The aim of the study was to examine the change in serum IgG levels measured by ELISA units to the immunodominant gp15 antigen as a consequence of the first episode of symptomatic cryptosporidial infection [1]. A total of 452 children were recruited over an 18month period starting in March 2002; 373 children completed the 3year followup. Fieldworkers visited each child twiceaweek to record any morbidity. Surveillance stool samples were collected every two weeks and diarrheal stool samples were collected with each episode of diarrhea [2]. The diarrheal stool samples were examined for the presence of Cryptosporidium spp. by microscopy and the positive samples were subjected to PCRRFLP for genetic characterization [3]. Fiftythree children in this cohort experienced a total of 58 episodes of confirmed cryptosporidial diarrhea, out of which 47 episodes were due to C. hominis (see details elsewhere [3]). For illustrative purposes, we used data from 40 children whose first episode of cryptosporidial diarrhea was due to C. hominis infection. For these 40 children we utlilized the results of ELISA testing in two surveillance stool samples collected before and after the child's first episode of cryptosporidial diarrhea. The original data are provided in supplemental material, which include information on IR values, sampling date and child's age (see Additional file 1). The details on measuring serum IgG levels to the gp15 antigen and normalization of ELISA units can be found elsewhere [1, 4]. For the purpose of this study, serum IgG levels are used as a measure of immune response.
The main objective for the performed statistical analysis is to derive inferences from a change in the immune responses measured in ELISA units at those two time points. In statistical terms we aim to detect the difference in the markers of immune responses in a study with a prepost design delivering two repeated measurements for each subject.
In this tutorial we use the following notations: Y _{ i }  values for immune responses for i child; each Y _{ i } consists of two values: Y _{ t1 } first measurement and Y _{ t2 } second measurement, where t _{1}  time of first measurement; t _{2}  time of second measurement. A degree of change on an individual level is defined in three ways: as an absolute difference, ΔY _{ i } = Y_{ t2 } Y_{ t1}, an absolute difference of logtransformed values, ΔY _{ i } = lnY_{ t2 } lnY_{ t1 }and logfold change, ΔY _{ i } = ln(Y_{ t2 }/Y_{ t1}). We also specify t_{ E } as the time of the event of interest. Additional relevant information to the presented illustration includes age at measurement and date of measurements. Sections below demonstrate the importance of this information in better understanding the variability in immune responses.
Results and Discussion
Traditional Statistical Approach
A traditional statistical approach to a prepost comparison in a controlled experiment consists of four steps. In Step 1, we examine the distributions of measures at both time points: pre and postevent, and provide summary statistics, e.g. mean, median, standard deviation, and interquartile range separately for pre and postevent measurements. In Step 2, we test for potential outliers, transform original values if necessary, and impute missing values if appropriate. In Step 3, we estimate the change for each subject, and again, provide summary statistics for the difference between pre and postevent measurements. In Step 4, we test the significance of change using parametric or nonparametric tests.
Summary statistics for the pre & post event immune response (IR)
Untransformed  Natural log transformed  

Pre  Post  Pre  Post  
Mean  54.9  171.2  3.56  4.95 
Standard deviation  87.9  117.1  0.86  0.65 
95^{th} percentile  122.5  478.9  4.81  6.17 
75^{th} percentile  57.9  196.1  4.06  5.28 
50^{th} percentile  38.6  135.9  3.65  4.92 
25^{th} percentile  20.1  101.9  2.99  4.63 
5^{th} percentile  8.86  36.6  2.14  3.63 
Interquartile range  37.8  94.1  1.06  0.65 
Skewness  5.09  1.53  0.48  0.15 
Kurtosis  30.1  4.97  4.29  3.02 
A histogram of the observed absolute differences, ΔY _{ i } = Y_{ t2 } Y_{ t1 }(Figure 2B), describes the pattern of change. The histogram provides a clear depiction of the predominant direction of change: the majority of children exhibited an increase in IR values (except for one case with a negative value); the difference between post and pre values ranges between (435.0 and 459.4), 50% of children exhibited a change of 104.4 units; 25% of children exhibited a change less than 42.4, 75% of children had a change less than 161.6, yielding an IQR of 119.2 units of IR. On average, there was an IR increase of 116.2 ± 114.3. Testing if the average change differs from zero, yields a tvalue of 5.095 that corresponds to a pvalue of < 0.001.
However, the observed skew in the distribution warrants the use of appropriate transformation and nonparametric tests that alleviate the effect of skew toward high IR values and potential outliers. In our case, a nonparametric Wilcoxon signedrank test also yields a very low pvalue of < 0.001. Next, we transform the IR data using natural log and produce boxplots for preevent and postevent IR data (Figure 2C). The summary statistics for transformed data are presented in Table 1. The transformed data has less skew as indicated by the boxplot and the coefficients of skewness and kurtosis in Table 1. The superimposed distribution line (in Figure 2D) offers an insight on how close the observed differences are to the normal distribution. On average, the absolute difference, DY _{ i } = lnY_{ t2 } lnY_{ t1}, was 1.38 ± 0.97. Testing if the average change differs from zero, yields a tvalue of 9.037 that corresponds to a pvalue of < 0.001. A better way to present the expected difference is to express it as a percent increase together with the 95%confidence interval: so, in our example the average change is 38% and 95% CI is [7.5%; 69.5%].
This standard analysis provides a first glance at the nature of change in the outcome of interest. The succinct form of visuals and tables offers a comprehensive overview of results that are clearly presented to support the study conclusion of a significant increase in IR following a diarrheal episode. In this analysis we assume that the mean and variance of the change are reasonably constant throughout the range of preevent values. We also assume that neither skew nor outliers affect the estimate of the change and the relationship between preand post event values. However, to reach meaningful conclusions these assumptions should be further explored.
Further Explorations
There is more to the behavior of individual differences that deserves further attention. In general, we assume that high postevent values simply reflect the magnitude of the immune response in that particular child. However, it is plausible that the degree of change in the immune response might depend on the baseline value. We may suspect that children with high preevent values tend to have high postevent IR values, but the relative increment would be small. Detection of such a relationship is an important finding because it highlights the heterogeneity in the observed change.
Cases with atypical behavior are often considered to be outliers and deserve special treatment. It is possible that their effect on the overall results is disproportionally strong and might bias our conclusions. Such bias has to be carefully examined. For example, Figure 2 shows that the highest EU value at baseline is associated with the largest decline in EU values: one child had an unusually high IR value before the event of interest, which later declined to approximately average IR values at the second time point. We consider this point to be an outlier based on the fact that it is biologically implausible to have a decrease in immune response following an event, unless the child had a prior exposure to the pathogen that has gone undetected. This outlier is marked in Figure 2. Now we illustrate how the effect of an outlier on the relationships between change and baseline values can be examined.
Let us flag and remove the suspicious value first noted when we plotted the baseline values in Figure 2 (marked as "outlier"). We intentionally did not emphasize the outliers for post measures in Figure 1 because the log transformation eliminates their effect [6]. Then, we recalculate the correlation between pre and postevent values in the sample without the case exhibiting unusual behavior (Figure 3). Once again, the overall correlation is weak and nonsignificant, although the estimates are getting closer (Pearson's r = 0.247, pvalue = 0.129; Spearman's ρ = 0.287, pvalue = 0.076). This observation supports the earlier statements that the pre and postevent IR's are not correlated.
For Figure 4A, the difference in correlations for actual values (Pearson's r = 0.279, pvalue = 0.816; Spearman's ρ = 0.633, pvalue < 0.001) is likely to be driven by the outlier. After removing the outlier the correlation coefficients become very high and similar (Pearson's r = 0.887, pvalue < 0.001; Spearman's ρ = 0.761, pvalue < 0.001). For the logtransformed data (Figure 4B) the correlations are low for a complete data set (Pearson's r = 0.278, pvalue = 0.082; Spearman's ρ = 0.147, pvalue = 0.366) as well as for the data with an outlier removed (Pearson's r = 0.125, pvalue = 0.449; Spearman's ρ = 0.080, pvalue = 0.630). This confirms the early statement that the change in actual values depends on IR and is scalesensitive.
These findings suggest that the immune response might exhibit more complex behavior than we originally postulated. We can suspect that the study group is heterogeneous and it is quite likely that some important factors, or interactions, must be considered in order to better understand the nature of the observed relationship. The child's time of measurement relative to time of exposure and infection, and the infecting dose are potential candidates for such factors.
In this section, we illustrated how one can test the hypothesis that children with high postevent values tend to have high preevent IR values. We also highlighted the importance of better understanding these associations in order to reach valid conclusions. In planning the prepost experiments in human cohorts, it is essential to collect information on parameters that can identify potential interactions and/or effect modification that might affect the degree of change. There are also a number of techniques that can further improve a statistical analysis protocol, including statistical tests for the difference in correlation coefficients.
Flaws of a Traditional Scheme: Assumption of Time Irrelevance
The traditional scheme, although simple and familiar, might be inadequate for "reallife" settings in measuring human immune response. The major flaws are in the basic assumptions that might be valid in experimental conditions but not in "field" studies. Typically, in experimental settings or in clinical settings, the time of measurement, the dose of exposure, and the timing of exposure are well controlled. In "field" or observational studies, the time of measurement, even in a wellrun cohort, is subject to fluctuations, which have to be carefully considered in the analysis. Moreover, the "time of event"  in our case the time of exposure to infection with a specific pathogen  could not be predicted (unless performed in volunteer studies). Such features do not make human studies any less precise or valuable, but they do require serious attention to details and the proper selection of statistical techniques.
One of the major problems of traditional analysis is the assumption of time irrelevance. Figure 2 adequately summarizes the change in individual measures. However it implies that the measurements were taken at the same time for all pre or post event samples. It also may imply that the time of measurements is irrelevant for the purpose of an analysis. The use of an absolute difference of pre and postevent measures, as a characteristic of change, implies that the observed change is solely due to the event of interest. The duration between pre and postevent measures, even if specified, is often assumed to be irrelevant to the observed magnitude of change. In this section, we illustrate that the time of measurement is not a trivial matter and must be specifically considered.
Summary statistics for the timing of pre & post event serum sample collection (IR)
Pre event  Post event  

Mean (SD)  7.7  8.3 
Standard deviation  5.9  4.9 
95^{th} percentile  18.7  17.6 
75^{th} percentile  10.1  11 
50^{th} percentile  5.9  6.9 
25^{th} percentile  3.4  4.8 
5^{th} percentile  1  2.4 
Interquartile range  6.6  6.2 
In observational studies the consideration of timing can be quite important for reaching valid conclusions. In the simplest cases and when the mentioning of duration between pre and post measurements contributes to a better understanding of a process, the change in IR can be easily adjusted. This can be done by presenting the average change per unit of time, e.g. days, weeks, or months. In our example, the summary statistics for IR increase, presented in Figure 2B, will change from its absolute value of 116.2 ± 114.3 to 8.3 ± 9.0, reflecting a weekly rate of change. This is a more accurate representation of overall change in IR, and is suitable if the assumption that the rate is constant over time for each child is correct. However, it is plausible that the magnitude of IR depends on when the sample was taken after an infection, or in other words, depends on the time elapsed since the time of exposure to a pathogen or of onset of symptoms. To explore this phenomenon, we shall perform a proper analysis.
Detection of Important Features in Immune Response Dynamics
It is well known that, in general, immune response to an antigen might exhibit a detectable increase after an antigenic challenge, reach its maximum and then decline to a baseline level or continue to be slightly elevated for some time. Such behavior is characteristic for nonlinear dynamics and requires caution if traditional statistical techniques are attempted.
This analysis helps us to define a curve without forcing a specific shape. If the episode timing is accurate and unbiased, one can model this process accounting for a discrete step with a curve consisting of two segments: one for a preevent period and another for a postepisode period, so a step will be allowed at the event. In our case, the timing of an episode is selfreported and might contain some delay, which is potentially reflected by a rise prior to the episode.
This analysis helps to identify important features of immune response dynamics: general stability of immune response before an episode and a nonlinear dynamic of response over time with a pronounced peak followed by a decline to a baseline level. These findings have important implications. For example, they might help to optimize future study designs by better targeting the time period that captures the best immune response.
Age as a Factor of Immune Response
Immune response can vary with age. The standard analysis can not explicitly account for potential differences in age. Even in the most closely observed study cohorts, the age at which a child contracts an infection cannot be controlled. To better understand how chronological age affects the immune response, we have to perform an analysis with an intention to detect the association between IR and the age at measurements.
Using regression coefficients as applied to pre and post measurements we are able to estimate the overall magnitude of an increase in IR and the increase in IR associated with one unit of age. Regardless of age after an episode of diarrhea, the IR is 4.2 times higher compared to the IR before an episode. Both preevent and postevent IR values depend on the child's age: on average, in one year (50 weeks) IR increases by 67.8% (95% CI: 62.3%, 73.2%) in baseline measurement and by 22.3% (95% CI: 19.2%, 25.5%) in postepisode measurements. We then confirm these results by removing an outlier and also by using a generalized linear model adapted for the logtransformed values and a Gaussian error structure (data not shown).
This analysis shows strong relationships associated with age at first encounter with an infectious agent, helps to determine how age contributes to overall increase in IR, and identifies critical features in IR related to time of measurements.
Environmental Exposure as a Factor of Immune Response
Diarrheal infections are known for their seasonal behaviors: some peak in hot summer months, some peak in cold dry winter months [12–14]. Environmental factors with seasonal effects drive the probability of exposure to pathogens, affect concentration and pathogenicity, modify susceptibility to infections; and have to be considered whenever it is possible [15]. It is rare that, in a large scale population study, such as a birth cohort, or even in a well run clinical investigation, the enrollment of study subjects can be done instantaneously or during a very short time interval. This means that during the study, subjects can be exposed to different environmental conditions, which can affect the time of an episode and modify the IR values obtained before and after an event of interest.
Summary statistics depicting the distribution of cases in different age groups and the change (logfold) in their immune response (IR)
Age (in months)  No. of cases  Logfold change in IR: Mean (SD) 

06  4  1.9 (0.5) 
612  13  1.8 (1.8) 
1218  9  1.1 (0.6) 
1824  6  1.3 (0.7) 
2430  7  1.1 (1.5) 
3036  1  2.9 
Conclusions
Human immunology is a growing field of research in which experimental, clinical, and analytical methods of many life science disciplines are utilized. Classic epidemiological study designs, including observational longitudinal birth cohort studies, offer strong potential for gaining new knowledge and insights into immune response to pathogens in humans. However, rigorous discussion of methodological issues related to designs and statistical analysis that are appropriate for longitudinal studies is lacking.
In this article, we addressed key questions of quality and validity of traditional and recently developed statistical tools applied to measures of immune responses and present examples that provide compelling evidence for the complexity of "field" sampling and the need for thorough examination of data originating from studies with a prepost design. For this purpose we used data on humoral immune response associated with the first cryptosporidial diarrhea in a birth cohort of children residing in an urban slum in south India. The main objective of the statistical analysis was to detect the difference and derive inferences for a change in IR measured at two time points, before (pre) and after (post) an event of interest. We illustrated the use and interpretation of analytical and data visualization techniques including generalized linear and additive models, datadriven smoothing, and combinations of box, scatter, and needleplots. We demonstrated that a detailed analysis allows us to describe new features of immune response to a pathogen in observational studies in humans. We borrowed our example from a birth cohort study, however the prepost design is a widely used approach and the challenges and application of the presented methodology is not limited to the birth cohort setting. In our example, the event of interest was a diarrheal episode, but it could be any specific detectable episode, e.g. exposure, infection, symptom, sign, disease, vaccination, practically any procedure or condition that might affect immune response.
The obtained results have very important implications for future study designs and analyses, including considerations for scheduling postevent sampling. The demonstrated techniques can be adapted for longitudinal studies when multiple measurements are taken from study subjects. For instance, studies on immune response following vaccination (similar to [19, 20]) will benefit from the proposed strategy of statistical analysis by accounting for the longitudinal nature of data collection.
We emphasized the need for analytical rigor in human immunology with respect to the use of statistical models as well as their tremendous capability to deal with issues related to the heterogeneity of immune response in humans and the complexity of research settings in "field" studies. Interplay between maternally acquired antibodies, prior exposure, time from exposure, and halflife of antibodies of different classes can all influence the development and duration of immune response in studies of vaccination and natural infection. We illustrated how the assumption of time irrelevance can be handled in a study with a prepost design and how one can study the dynamics of IR in humans by considering the timing of response following an event of interest and seasonal fluctuation of exposure by proper alignment of time of measurements. This alignment of calendar time of measurements and a child's age at the event of interest can help to explore interactions between immune response, seasonal exposures and age at first infection, and ultimately the intrinsic features of infection dynamics.
We also emphasized the need for a better understanding of theoretical assumptions that form the foundation of proper application of statistical methods and the validity of such assumptions in the context of designed experimental settings. Even in a simple preexperimental "one group prepost design" with no additional samples for comparison it is extremely important to carefully examine underlying assumptions. The results of these studies can sufficiently inform researchers as to how to properly design the next steps and avoid costly mistakes. Furthermore, the intent of any life sciences study is to contribute to general knowledge so a thoughtful approach to each analytical step is necessary.
In human immunology (as in many other fields), it is widely accepted that experimental methods, e.g. randomized clinical trials, are the "gold standard" for causal inferences and measuring performance and the value of observational studies is belittled. While the shortcomings of nonexperimental, or quasiexperimental, designs have been frequently discussed [21], a well conducted and thoughtfully evaluated prepost study can provide highly valuable information, be rich in scope and can profoundly capture the process being investigated [22]. An effort should be made in order to supplement observational studies with powerful analytical tools similar to designed for clinical trials [23]. The complexity of multifaceted informational structures in data collected from large observational studies requires a thoughtful analytical plan, careful assessment of potential interactions, effect modifications, and confounding, selection of statistical methods and models appropriate to underlying assumptions, and an integrative interpretation of modeling results.
List of Abbreviations
 CI:

confidence interval
 IR:

immune response measure
 EU:

ELISA (Enzyme Linked Immune Sorbent Assay) units
 PCRRFLP:

polymerase chain reactionrestriction fragment length polymorphism
 GLM:

generalized linear model
 GAM:

generalized additive model
 ttest Student's t :

test.
Declarations
Acknowledgements
This work was supported by PHS grants R03 TW2711 "Molecular epidemiology of cryptosporidiosis in India", NIAID R01 AI072222 "Immune Response to Cryptosporidium in a Birth Cohort of Children of South India" (PI  Ward); CDC/IndoUS Concept "Environmental Indicators for Diarrheal Infections in South India" (PI  Naumova). RS was supported by the Global Infectious Disease Research Training Grant (D43TW007392; PI  Kang). The data used in this tutorial was collected from a study supported by the Wellcome Trust Trilateral Initiative for Infectious Diseases (Grant No.: 063144). Authors wish to express their gratitude to Drs. Janet Forrester, Bertha Estrella, Yuri Naumov, and Stefan CollinetAdler for their thoughtful comments and suggestions.
Authors’ Affiliations
References
 Ajjampur SS, Sarkar R, Allison G, Banda K, Kane A, Muliyil J, Naumova E, Ward H, Kang G: Serum IgG response to Cryptosporidium immunodominant antigen gp15 and polymorphic antigen gp40 in children with cryptosporidiosis in South India. Clin Vaccine Immunol. 2011, 18 (4): 633639. 10.1128/CVI.0046410.View ArticlePubMedPubMed CentralGoogle Scholar
 Gladstone BP, Muliyil JP, Jaffar S, Wheeler JG, Le Fevre A, IturrizaGomara M, Gray JJ, Bose A, Estes MK, Brown DW, et al: Infant morbidity in an Indian slum birth cohort. Arch Dis Child. 2008, 93 (6): 479484. 10.1136/adc.2006.114546.View ArticlePubMedGoogle Scholar
 Ajjampur SS, Gladstone BP, Selvapandian D, Muliyil JP, Ward H, Kang G: Molecular and spatial epidemiology of cryptosporidiosis in children in a semiurban community in South India. J Clin Microbiol. 2007, 45 (3): 915920. 10.1128/JCM.0159006.View ArticlePubMedPubMed CentralGoogle Scholar
 Khan WA, Rogers KA, Karim MM, Ahmed S, Hibberd PL, Calderwood SB, Ryan ET, Ward HD: Cryptosporidiosis among Bangladeshi children with diarrhea: a prospective, matched, casecontrol study of clinical features, epidemiology and systemic antibody responses. Am J Trop Med Hyg. 2004, 71 (4): 412419.PubMedGoogle Scholar
 Bland JM, Altman DG: Comparing methods of measurement: why plotting difference against standard method is misleading. Lancet. 1995, 346 (8982): 10851087. 10.1016/S01406736(95)917489.View ArticlePubMedGoogle Scholar
 Hadi AS, Imon AHMR, Werner M: Detection of outliers. Wiley Interdisciplinary Reviews: Computational Statistics. 2009, 1 (1): 5770. 10.1002/wics.6.View ArticleGoogle Scholar
 Gijbels I, Prosdocimi I: Loess. Wiley Interdisciplinary Reviews: Computational Statistics. 2010, 2 (5): 590599. 10.1002/wics.104.View ArticleGoogle Scholar
 Frost FJ, Muller T, Craun GF, Fraser D, Thompson D, Notenboom R, Calderon RL: Serological analysis of a cryptosporidiosis epidemic. Int J Epidemiol. 2000, 29 (2): 376379. 10.1093/ije/29.2.376.View ArticlePubMedGoogle Scholar
 Hastie T, Tibshirani R: Generalized additive models. 1990, London; New York: Chapman and Hall, 1Google Scholar
 Wood SN: Generalized additive models: an introduction with R. 2006, Boca Raton, FL: Chapman & Hall/CRCGoogle Scholar
 McCullagh P, Nelder JA: Generalized linear models. 1989, London; New York: Chapman and Hall, 2View ArticleGoogle Scholar
 Jagai JS, Castronovo DA, Monchak J, Naumova EN: Seasonality of cryptosporidiosis: A metaanalysis approach. Environ Res. 2009, 109 (4): 465478. 10.1016/j.envres.2009.02.008.View ArticlePubMedPubMed CentralGoogle Scholar
 Levy K, Hubbard AE, Eisenberg JN: Seasonality of rotavirus disease in the tropics: a systematic review and metaanalysis. Int J Epidemiol. 2009, 38 (6): 14871496. 10.1093/ije/dyn260.View ArticlePubMedGoogle Scholar
 Naumova EN, Jagai JS, Matyas B, DeMaria A, MacNeill IB, Griffiths JK: Seasonality in six enterically transmitted diseases and ambient temperature. Epidemiol Infect. 2007, 135 (2): 281292. 10.1017/S0950268806006698.View ArticlePubMedGoogle Scholar
 Grassly NC, Fraser C: Seasonal infectious disease epidemiology. Proceedings of the Royal Society B: Biological Sciences. 2006, 273 (1600): 25412550. 10.1098/rspb.2006.3604.View ArticlePubMedPubMed CentralGoogle Scholar
 Gladstone BP, Muliyil JP, Jaffar S, Wheeler JG, Le Fevre A, IturrizaGomara M, Gray JJ, Bose A, Estes MK, Brown DW, et al: Infant morbidity in an Indian slum birth cohort. Arch Dis Child. 2008, 93 (6): 479484. 10.1136/adc.2006.114546.View ArticlePubMedGoogle Scholar
 Banerjee I, Ramani S, Primrose B, Moses P, IturrizaGomara M, Gray JJ, Jaffar S, Monica B, Muliyil JP, Brown DW, et al: Comparative study of the epidemiology of rotavirus in children from a communitybased birth cohort and a hospital in South India. J Clin Microbiol. 2006, 44 (7): 24682474. 10.1128/JCM.0188205.View ArticlePubMedPubMed CentralGoogle Scholar
 Ajjampur SSR, Gladstone BP, Selvapandian D, Muliyil JP, Ward H, Kang G: Molecular and spatial epidemiology of cryptosporidiosis in children in a semiurban community in South India. J Clin Microbiol. 2007, 45 (3): 915920. 10.1128/JCM.0159006.View ArticlePubMedPubMed CentralGoogle Scholar
 Sun Y, Bian C, Xu K, Hu W, Wang T, Cui J, Wu H, Ling Z, Ji Y, Lin G, et al: Immune protection induced on day 10 following administration of the 2009 A/H1N1 pandemic influenza vaccine. PLoS One. 2010, 5 (12): e1427010.1371/journal.pone.0014270.View ArticlePubMedPubMed CentralGoogle Scholar
 Viana PO, Ono E, Miyamoto M, Salomao R, CostaCarvalho BT, Weckx LY, de MoraesPinto MI: Humoral and cellular immune responses to measles and tetanus: the importance of elapsed time since last exposure and the nature of the antigen. J Clin Immunol. 2010, 30 (4): 574582. 10.1007/s1087501094207.View ArticlePubMedGoogle Scholar
 Harris AD, Bradham DD, Baumgarten M, Zuckerman IH, Fink JC, Perencevich EN: The Use and Interpretation of QuasiExperimental Studies in Infectious Diseases. Clinical Infectious Diseases. 2004, 38 (11): 15861591. 10.1086/420936.View ArticlePubMedGoogle Scholar
 Concato J: Observational versus experimental studies: what's the evidence for a hierarchy?. NeuroRx. 2004, 1 (3): 341347. 10.1602/neurorx.1.3.341.View ArticlePubMedPubMed CentralGoogle Scholar
 Liang KY, Zeger SL: Longitudinal data analysis of continuous and discrete responses for prepost designs. Sankhya: The Indian Journal of Statistics, Series B, Part 1. 2000, 62 (Series B, Part 1): 134148.Google Scholar
 The prepublication history for this paper can be accessed here:http://www.biomedcentral.com/14712288/12/1/prepub
Prepublication history
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.