- Research article
- Open Access
Informative graphing of continuous safety variables relative to normal reference limits
BMC Medical Research Methodology volume 18, Article number: 40 (2018)
Interpreting graphs of continuous safety variables can be complicated because differences in age, gender, and testing site methodologies data may give rise to multiple reference limits. Furthermore, data below the lower limit of normal are compressed relative to those points above the upper limit of normal. The objective of this study is to develop a graphing technique that addresses these issues and is visually intuitive.
A mock dataset with multiple reference ranges is initially used to develop the graphing technique. Formulas are developed for conditions where data are above the upper limit of normal, normal, below the lower limit of normal, and below the lower limit of normal when the data value equals zero. After the formulae are developed, an anonymized dataset from an actual set of trials for an approved drug is evaluated comparing the technique developed in this study to standard graphical methods.
Formulas are derived for the novel graphing method based on multiples of the normal limits. The formula for values scaled between the upper and lower limits of normal is a novel application of a readily available scaling formula. The formula for the lower limit of normal is novel and addresses the issue of this value potentially being indeterminate when the result to be scaled as a multiple is zero.
The formulae and graphing method described in this study provides a visually intuitive method to graph continuous safety data including laboratory values, vital sign data.
Graphic presentations have long been recognized as a useful method to present clinical trial data [1, 2]. This is particularly true for safety data because clinical studies are not typically designed to inferentially test safety and tolerability data. The International Conference on Harmonization (ICH) guidance to industry on Statistical Principles for Clinical Trials recommends graphic inspection for safety and efficacy, data, including dose-response relationships .
Accurately interpreting the graphs of safety data may be challenging because of differences in laboratory reference ranges between testing sites, genders, and age groups. For example, the reference range for plasma alkaline phosphatase concentrations varies between age, gender, and testing center. The reference ranges of many pediatric laboratories and vital sign tests vary considerably, particularly before adolescence, so interpretation of the graphical display from a clinical trial of one or more year’s treatment duration in the very young can be particularly challenging. Graphing the actual values can also result in a misleading mixture of normal and abnormal results in the regions of the upper and lower limits of normal because of overlapping differences in these ranges. These differences are sometimes due to variances in assay techniques or demographics of the reference population between sites.
Several studies recognize these issues and suggest using multiples of the reference range limits instead of the actual values, although these descriptions are typically limited to demonstrating values relative to the upper limit of normal (ULN). When this approach is applied to data with abnormal values above and below normal range, the normal and abnormal low values are ‘compressed’ between 1 and 0 multiples of the ULN, while the abnormal high values have, at least theoretically, no upper limit . This is not usually concerning when working with data from biomarkers of cell injury, such as lactate dehydrogenase (LDH), which can have multiples in the thousands over the ULN and where a low value is not clinically concerning. However, other analytes, such as serum glucose or hematocrit, often have abnormal values higher than the ULN and below the LLN. One approach to the problem of displaying values above and below the normal range is to plot the standard deviations of the sample mean; however, one is often testing a sample where almost all of the subjects have abnormal values, and so the result would look almost the same as if a similarly homogeneous population of normal subjects was tested.
The objective of this study is to develop a method of graphing continuous safety data in a manner so as to address the issue of compression, particularly with datasets having large numbers of data below the LLN and for data with abnormal values both above the ULN and below the LLN. The problems encountered with graphing the actual values using standard techniques are first demonstrated and then formulas are derived to graph data in a manner that addresses scalability above and below the normal range and to produce graphs that are visually intuitive.
Datasets used for methods development and analyses
Methods for graphing were developed using a mock labs data set (MDLD.JMP) generated in the software program JMP® (v.13, SAS Institute). Lab values were produced using a formula-based random numbers generator between numeric ranges seen with actual testing of an analyte that may have abnormal high and low values, such as transferrin. Additional values of 0 (zero) were substituted for demonstration purposes. Demographic column variables such as unique subject identifier (USUBJID), Testing Site (SITE), Planned Treatment (TRTP), and Visit sequence (VISIT) are also included in the dataset. The mock dataset is constructed so that subjects assigned to be tested randomly at each of the four sites during the course of the trial. Columns are created for the ULN and lower LLN limit of normal reference ranges for each value (RESULT) based on the testing site at the time of the laboratory blood draw based on those publically available for transferrin from the 4 testing sites (SITE) (Table 1). An abnormal flag column (ABNFL) is added to the dataset to designate the RESULT as being normal (N), abnormally high (H), or abnormally low (L).
Graphical display of untransformed lab datasets
Graphs are first created for the untransformed data using the variable for time (e.g., VISIT in MDLD.JMP) as a continuous variable for the X-axis. The RESULT is a continuous dependent variable on the Y-axis. For the purposes of demonstration, lines for the normal reference limits are added to the Y-axis designating the ULN and LLN for each of the laboratory testing sites. Colored row states are created designating H as blue + signs, N as orange dots, and L as black inverted triangles.
Graphic inspection of untransformed data
Figure 1 is a presentation of RESULTS from the MDLD dataset by visit month with site-specific reference lines for the ULN and LLN. At the lower magnification, one can appreciate that values below the LLN are relatively compressed compared to those above the ULN (Fig. 1a). One can appreciate at higher magnification that the region of the LLN reference limits contains a mixture of normal and abnormally low values (Fig. 1b); this may be seen at the level of the values around the ULN lines as well. These issues may be exacerbated in trials with multiple sites (e.g., if sites use different assays), with pediatric patients, and where both genders are represented (e.g., if there are gender-specific reference ages).
Derivation of formulas to transform data for graphing
The formulae needed to graph the possible scenarios (L, N, H) are found in the following set of equations:
Equation 1 Multiples of ULN (xULN in the MDLD dataset) for graphing “H” values – This is the standard formula for ULN
Equation 2 – Normal results (NLRES), for graphing values equal to or falling between the ULN and LLN. This is a standard formula for scaling or normalizing, results between 2 numbers [5, 6], that has been adapted for this case where the boundaries are the ULN and LLN
Equation 3 - Multiples of LLN (xLLN) for graphing “L” values – This is the ‘multiplicative inverse’ of the ULN-type formula, so the results graphically fall below − 1, where the LLN is situated
Equation 4 - Multiples of LLN (MI) for graphing “L” values where the RESULT is equal to 0 – A special case of the xLLN formula is needed because the RESULT is in the denominator of the MI formula and in the case where the result is 0, the MI would be undefined, or ∞. The Multiplicative Inverse to be used when the result equals 0 is designated as MI 0 ; the final formula is presented below and its derivation follows
MI0 will be some distance MIA→0 beyond MImax, the multiplicative inverse of the abnormal value, A, with the greatest distance from its LLN (see Eq. 3). The variables in this last statement may be represented as
When these are added together, they equal the multiplicative inverse of 0.
Equation 4.2 simplifies algebraically to the following:
Substituting back the terms for A and B, the final formula is demonstrated.
Formulae needed to graph transformed lab datasets in JMP
Graphs are created for the transformed data using the methodology described for the untransformed data except that the continuous dependent variable on the Y-axis is a conditional formula (e.g., If, Then…) containing the formulae for the following:
Multiple of the ULN for RESULTS that are abnormally high,
Normal RESULTS scaled between 1 and − 1,
Multiple or multiplicative inverse of the LLN for RESULTS that are abnormally low but greater than 0, and for the
Multiple or multiplicative inverse of the LLN for RESULTS that are abnormally low but equal to 0. Reference lines are added to the Y-axis at (0, 1) and (0, − 1) to designate 1× multiple of the ULN and LLN, respectively.
Figure 2 presents these formulae as they appear in the JMP Formula Editor. The steps necessary to arrive at the formula for the MI0 in the JMP dataset include:
Create columns for the RESULT and corresponding LLN
Create a Column MI using the formula LLN ÷ RESULT
Create a Column Max M using the formula Col Maximum(M). This is done using Col Maximum from the Statistical function of the Functions (grouped) dialogue box and designating Column M as the variable of operation.
A column (Row of Max M) with the conditional formula, IF [M] = [MAX M] → [ROW()] specifies the row containing the maximum M for the next step. All of the cells in this column will appear to be missing values, except the rows corresponding to the maximum M value.
The formula for MI0 is written as one of the scenarios (last line, Fig. 2) for the conditional function that will be graphed on the Y Axis. The MIo formula makes use of Subscript from the Row function of the Functions (grouped) dialogue box to exclusively use the RESULT and LLN of the maximum multiple of the LLN.
Graphical display of transformed data
Figure 3 displays the results when graphing continuous variables where values that are both abnormally low and high are presented with a similar scale. Normal values are displayed in a linear distribution between the ULN and LLN. The black arrows pointing to purple inverted triangles indicate the locations where the multiplicative inverse of the LLN is located for RESULTS of zero based on the equation in Eq. 4.3. The black arrow pointing to the light blue inverted triangle indicates the location of the multiplicative inverse, MImax, described in Eq. 4.1a. This is the point that is used to derive the position of the MI0, as demonstrated in Eq. 4.
Advantages of graphic presentation of safety data
Graphic presentation of safety variables provides a means to enhance the accuracy of signal detection. Evidence for this has been reviewed in a recent publication by Michard . Businesses, such as aeronautics, have long known this, designing key displays with graphics rather than numeric displays. The advantage of graphical presentation has been systematically studied with respect to vital sign data in the disciplines of cardiology [8, 9], anesthesiology , and critical care medicine . Evaluation of clinical data in graphic form or a combination of numeric and graphic data has been shown to produce improved data interpretation speed and accuracy in several clinical scenarios. In one study of home blood pressure monitoring, this difference in monitoring resulted in differences in patient management, as measured by medications used. While definitive evidence was not found that graphic display could have a meaningful effect on mortality, several studies mentioned above have demonstrated significant improvements in the detection of myocardial ischemia or acute coronary syndrome, which can be fatal.
An unmet need in graphical presentation of data
Potential discrepancies and deficiencies in standard numeric reporting of clinical safety data values have been long recognized. In the 1970’s, Duboff reported creatine kinase values for members of a family with malignant hyperthermia that differed based on the testing site . In this same period of time, the College of American Pathologists’ Enzyme Survey demonstrated that results of liver enzyme tests varied by site, presumably because of differences in instruments, reagents, and assay conditions [13, 14]. The findings from these studies in the 1970’s coincided with a revolution in the use of graphical presentation of data, prompted largely by the work of Cleveland [1, 15, 16] and similar biometricians.
Recent attention to the use of graphics in the presentation of safety data has brought refinements [17, 18] including the use of individual patient graphic profiles ; however, limitations and misperceptions still exist.
Rodbard discussed potential issues in perception when viewing data graphed for analytes such as glucose, which may have abnormal data that are both above the ULN and below the LLN. Despite being potentially more critical in the acute setting, hypoglycemic values may be overlooked when presented graphically because, in contrast, hyperglycemic values can be so relatively large that the appearance of the low values does not appear meaningfully different than those that are normal when graphed using standard methods . While the concept of compression discussed by Bottger and Balzer  relates more to distortions caused by two unequal stretching operations, the phenomenon described by Rodbard et al., seems to describe compression of abnormally low values relative to the scaling of those above the ULN.
Previous proposals for graphing safety data
Several authors have proposed alternative methodologies for graphing. The most commonly used is based on multiples of the ULN. This method has the most utility for monitoring plasma levels for enzyme biomarkers of tissue damage such as AST, ALT, and CPK. One of the original methods proposed “centrinormalized units” that divide the ULN by 100 and scale all results by this factor so the normal range would always be 100 (upper limit) to LLN*100/ULN (lower limit) . More recently, a system based on the ULN has been proposed as the method to evaluate drug-induced liver toxicity based on ‘Hy’s Law’ . A closely related alternative methodology proposes to use both multiples of the ULN and baseline in a statistical outlier technique to define the critical boundaries of toxicity methods . While the ULN-based methods have value in certain analyses, this methodology would not be useful when plasma analytes or vital sign measurements have values below the lower limit of normal. Normal values would also be scaled based on the ULN, which would make these data difficult to interpret.
Methods of graphing based on the standard deviation (SD) of data have also been proposed, where for example, data falling outside of 2  or 3  standard deviations of the mean would be considered abnormal. This method has the advantage of being able to described data above the ULN and LLN in a scale that is equivalent; however, the utility of this method is limited because not all sample populations are equivalent between hospital or lab testing sites and the population that comprises the mean ± 2 SD is not necessarily normal.
Rodbard  proposed use of a semilogarithmic plot to triple the percentage of the vertical axis allocated to values below the LLN, as is encountered in cases of hypoglycemia, while at the same time compressing the region of the graph containing hyperglycemic values. This method has value in expanding the detail for values below the LLN, yet it is not optimal for simultaneously visualizing data containing low, normal and high values.
Scaling of values with the proposed method
The primary objective in developing this method was to provide a means to graph data where values below the LLN were scaled the same as those above the ULN. The first step was relatively simple; transforming abnormally high values above the ULN is typically done by dividing the value by the ULN reference limit. The scaling of results below the LLN is not typically found in literature and maintaining a scalar, visually intuitive display is not as simple as the ULN. The multiple of LLN is multiplied by − 1 in the last step before graphing so that it occupies a position equidistant to normal values as those above the ULN; this final value is termed the multiplicative inverse. While the issue of zero in the denominator would not exist if the formula was configured the same as that of the ULN (e.g., RESULT/LLN), this relationship cannot be used because zero values would result in a multiple equal to zero, which would place the point in the middle of the normal values, which are scaled between 1 and − 1.
The calculation of the multiples of the abnormal low value below the LLN will be problematic when the RESULT equals zero, since this would make the multiplicative inverse an undefined number using the equation LLN/RESULT. This situation is not typically found with certain data, such as vital signs or chemistry laboratories, where having a result of zero is not typically observed. However, there are laboratories where zero is encountered (e.g., the absolute neutrophil count in profound neutropenia) and so the issue must be addressed. A RESULT of zero would intuitively have a multiple greater than any non-zero value in the column. An additional quantity would need to be added to the column maximum of the greatest multiple of the LLN equal to a ratio that is based on the LLN and the value remaining after the smallest result is subtracted from the LLN, as is demonstrated in the derivation of Eq. 4.
Limitations and concerns
Transformation of data almost always has potential issues with the audiences’ ability to comprehend the relationship of numbers when graphically displayed. Log transformation is one of the procedures most often used and it results in a display that may be uninterpretable by those outside of the scientific field. The method presented in this study has the advantage that it is visually intuitive; abnormal data of equal magnitude are equidistant from the normal reference limits. Most who routinely work with data are so accustomed to compression of results less than the lower limit of normal displayed in the Cartesian system relative to that above the upper limit of normal that visualization of the data in a more naturally expanded state will require familiarization. The advantage of this technique, particularly, the ability to look at data with multiple reference ranges in the same graph, will make such effort worthwhile. The method to graph points with a value of zero is not likely to be used often. Other techniques such as inclusion of a ‘zero-corrector’ (such as the use of ½ when an incidence is zero in the calculation of relative risk) may be simpler though this introduces error in the visual display.
A method to graph continuous safety variables is presented. This method addresses the issues that arise when graphing values that have different reference limits, such as sometimes seen with differences in gender, age, and testing site. The graphing method presents novel approaches to the plotting of normal values in graphs of multiples of the reference limits based on a classic scaling equation. The method also includes a novel approach to scaling the multiples of the abnormal low value below the LLN when the result is zero.
Greatest non-zero number below LLN used in the derivation formula
LLN used in the derivation formula
Abnormally high value
International Conference for Harmonization
Abnormally low value
Lower limit of normal
- LLNmax :
Lower limit of normal for the RESULTmax
MI used in the JMP column formula to derive MI0
- Max M:
MI of the RESULTmax
Mock labs data set
Milligrams per deciliter
- MI0 :
Derived multiplicative inverse of zero
- MIA → 0 :
Distance from MImax to MI0
- MImax :
Multiplicative inverse of the non-zero abnormal value, A, with the greatest distance from its LLN
Values equal to or falling between the ULN and LLN
- RESULTmax :
Non-zero number greatest magnitude below LLN, also designated as A
Upper limit of normal
Unique subject identification
Multiples of LLN
Multiples of ULN
Cleveland WS. The elements of graphing data. Monterey: Wadsworth Advanced Books and Software; 1985.
Powsner SM, Tufte ER. Graphical summary of patient status. Lancet. 1994;344(8919):386–9.
International Conference on Harmonization. E9 Statistical Principles For Clinical Trials; 1998. p. 39.
Bottger J, Balzer M, Deussen O. Complex logarithmic views for small details in large contexts. IEEE Trans Vis Comput Graph. 2006;12(5):8.
Strickland S. Dynamically Scale a Range of Numbers. FMForums Worldwide Filemake Community 2013 [cited 2015 December 30]; Available from: http://fmforums.com/topic/90562-dynamically-scale-a-range-of-numbers/
user25658, A. How to normalize data to 0–1 range? Cross Validated 2013 9/23/13 11/11/16]; Available from: http://stats.stackexchange.com/questions/70801/how-to-normalize-data-to-0-1-range
Michard F. Decision support for hemodynamic management: from graphical displays to closed loop systems. Anesth Analg. 2013;117(4):876–82.
Nimmermark MO, et al. The impact of numeric and graphic displays of ST-segment deviation levels on Cardiologists' decisions of reperfusion therapy for patients with acute coronary occlusion. J Electrocardiol. 2011;44(5):502–8.
Kabutoya T, et al. A home blood pressure monitor equipped with a graphic function facilitates faster blood pressure control than the conventional home blood pressure monitor. J Clin Hypertens. 2009;11(8):422–5.
Syroid ND, et al. Development and evaluation of a graphical anesthesia drug display. Anesthesiology. 2002;96(3):565–75.
Drews FA, Doig A. Evaluation of a Configural vital signs display for intensive care unit nurses. Hum Factors. 2014;56(3):569–80.
Duboff GS. Expressing results of enzyme assays. Clin Chem. 1974;20(3):2.
Lott JA, Gilmore BE, Massion CG. Lnterlaboratory survey of enzyme analyses: V. A simplified method of reporting Participants' data. Am J Clin Pathol. 1983;80:8.
Lott JA, Tholen DW, Massion CG. Determination of reference ranges for serum enzymes via a large Interlaboratory survey. A1ch Patho!LabMed-V. 1987;111:7.
Cleveland WS, Kleiner B, Warner JL. Robust statistical methods and photochemical air pollution data. Air Pollut Control Assoc. 1976;26(1):36–8.
Cleveland WS, McGill R. Graphical perception - theory, experimentation, and application to the development of graphical methods. J Am Stat Assoc. 1984;79(387):531–54.
Chuang-Stein C, Le V, Chen W. Recent advancements in the analysis and Presentaton of safety data. Drug Inform J. 2001;35:377–97.
Amit O, Heiberger RM, Lane PW. Graphical approaches to the analysis of safety data from clinical trials. Pharm Stat. 2008;7(1):20–35.
Ma H, et al. Graphical analyses of clinical trial safety data. In: Joint statistical meeting - statistical computing and statistical graphics. Salt Lake City: American Statistical Association; 2007. p. 42.
Rodbard D. A Semilogarithmic scale for glucose provides a balanced view of hyperglycemia and hypoglycemia. J Diabetes Sci Technol. 2009;3(6):7.
Gerstbrein HL, Lederer WH. Uniform presentation of enzyme determinations. J Am Med Assoc. 1974;227(3):1.
Administration, F.a.D. Guidance for industry drug-induced liver injury: premarketing clinical evaluation, D.O.H.A.H. Services, editor; 2009. p. 25.
Lin X, et al. Validation of multivariate outlier detection analyses used to identify potential drug-induced liver injury in clinical trial populations. - PubMed - NCBI. Drug Saf. 2012;35(10):11.
Miyamoto M, Shimizu H, Niwa M. Delta check system for detection and further Management of Laboratory Errors. Tokai J Exp Med. 1986;11(2):11.
Mayer M, Chou D, Eytan T. Unit-independent reporting of laboratory test results. Clin Chem Lab Med. 2001;39(1):3.
I thank Dr. Hylton Joffe for his critical reading of and suggestions for this manuscript.
Availability of data and materials
The data supporting this manuscript is found in the Summary Basis of Approvals at Drugs@FDA (http://www.accessdata.fda.gov/scripts/cder/daf/).
Ethics approval and consent to participate
Dr. Breder is an employee of the United States Food and Drug Administration. The views expressed in this manuscript represent the opinions of the author and do not necessarily represent the views of the FDA. The author does not have any financial interest in the SAS Institute® or any of its products.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Breder, C.D. Informative graphing of continuous safety variables relative to normal reference limits. BMC Med Res Methodol 18, 40 (2018). https://doi.org/10.1186/s12874-018-0504-z
- Multiplicative inverse
- Reference range