Medical records, interviews and self-administered questionnaires are frequently used sources of research data, thus many studies have assessed the concordance and reliability between these data collection methods. Although some researchers have considered medical records as the “gold standard” and the preferred data source over questionnaires, evidence indicates that neither source is completely accurate and that combining sources may result in a more reliable and complete data assessment [1–5]. Any method of data collection can introduce its own sources of measurement error. Questionnaire data, either from self-report or interview, have limitations such as recall bias, misinterpretation of the questions and degree of willingness to report. On the other hand, medical record data may also be limited by illegibility; incomplete, inaccurate or missing documentation; and limited availability of data elements [2, 6–8]. Furthermore, medical records are designed for clinical rather than research purposes. Thus, multiple challenges exist regarding what data to extract and how, especially when multiple investigators are involved if a standardized extraction protocol is not established [9, 10].

While published population-based studies have examined the concordance between medical records and questionnaire data for chronic medical conditions and its related symptoms [3, 4, 11–13], we did not find literature within the past decade that summarized and compared the reliability and concordance between these two data sources in an incarcerated population. Because we were collecting data for a research study and drawing inferences from these data which could have an impact on clinical care and policies (see ‘Study population’ below), it was essential to determine the extent to which various data sources were complete, available, and consistent. Therefore, the aim of this study was to assess the concordance between medical records and interview questionnaires for medical and behavioral conditions obtained from two maximum security correctional facilities.

We evaluated the concordance between medical records and interview questionnaires for medical and behavioral conditions in two incarcerated populations in New York State. Similar to the findings of Schofield et al. [17], we found that inmates were generally reliable respondents for health-focused surveys. Overall, our findings were consistent with previously published studies conducted in either a community or clinical settings; the prevalence of chronic medical conditions except for renal/kidney disease was higher in the medical records when compared to the questionnaires, which could be due to under-reporting in the interviews, as has been previously reported [13, 17–20]. Similarly, behavioral conditions are likely to have been under-reported in the medical records, especially for variables such as drugs and antibiotics. In contrast to a literature summary by Garber et al. [21], we found that interviews had just as good concordance as self-administered questionnaires when compared with medical records.

Like Okura, et al. [3], which was published almost a decade ago, and Malik et al. [19], a more recent study, we also found strong concordance between medical records and questionnaire responses for HIV and diabetes, which could indicate that most participants were aware of their diagnoses and willing to disclose that information [3, 12, 13, 19, 20].

Consistent with findings of Iversen, et al., Leikauf et al., and Tisnado, et al., we also found good concordance for reporting of asthma [5, 13, 18]. Hepatitis C, on the other hand, had a lower concordance level and was more likely to be reported by medical record, suggesting that participants were either unaware of their status or unwilling to report to the investigators.

In contrast to medical conditions, all behavioral conditions were reported more frequently in the interview questionnaires than in the medical records. Reports on history of any illicit drug use and marijuana use had the lowest kappa scores and the greatest difference between the two sources, perhaps because inmates are less likely to report drug use to health care providers during the physical examination than to the interviewers in fear of reprehension. Our kappa scores for current cigarette smoking and cocaine use were slightly lower than those reported in a previous study [19]. Since certain medical information was not up to date, the most recent reports of current cigarette smoking might not be representative of the inmate’s current smoking habits. A history of tattoos, on the other hand, had the best concordance and highest sensitivity/specificity in behavioral conditions. This could be explained by the fact that tattoos are noticeable and legal, thus inmates may not be wary of reporting them. Overall, interview questionnaires may be a better source of data for behavioral conditions than the medical record.

After gender stratification, no significant differences were found as compared to the gender non-stratified analysis; however, females did report much higher prevalence and concordance levels than males for all variables, also consistent with previous research [20, 22]. Since both facilities have similar medical care accessible to inmates, this difference could be due to actual higher prevalence of conditions in females, the fact that females may be more aware and health conscience than males, therefore more willing to share information, or that females tend to frequent medical unit more often than males. Further studies on gender differences should be conducted to clarify these distinctions.

This study had limitations and bias that could have affected our findings. The incarcerated population may not be generalizable to other populations. As previously reported, medical records are often incomplete, missing information, or not up to date [2, 7, 17, 18]. Specifically in this study, the medical records were handwritten, not electronic records like in the population-based studies, thus it was difficult to retrieve the necessary or, at time, accurate information. We did not record the length of time required to extract data from the medical records, but it varied considerably, depending upon the handwriting in the notes. Clearly, data extraction would be greatly facilitated in electronic medical records.

Because two research assistants were conducting the interviews and extracting from the medical charts, there could be variations in data collection processes and interpretation. However, this did not have any significant effect on the data analysis or results, since the medical record form was straightforward and both research assistants were trained to follow a standardized extraction protocol. Furthermore, studies have shown high kappa scores and percent agreement of intra- rater and inter-rater reliability for medical record extraction [10, 23, 24]. Most importantly, for many of the variables we examined, particularly the behavioral factors such as sexual practices, it was not possible to assess validity because there was no confirmatory ‘gold standard’. Nevertheless, our findings should provide some guidance as to when the medical record or self-report might be the most reliable data source.

Medical records and self-reports are often data sources used in research, thus it is essential for investigators to analyze the concordance between the two for any variables of interest. While our findings were similar to what has been previously reported for chronic medical conditions regardless of the dissimilarity in study population, the level of concordance between the two sources varied greatly depending on the variable. Hence, investigators should choose data sources and construct questionnaire forms carefully depending on the population and variables of interest.