Pharmacy sales data versus ward stock accounting for the surveillance of broad-spectrum antibiotic use in hospitals
BMC Medical Research Methodology volume 11, Article number: 166 (2011)
Antibiotic consumption in hospitals is commonly measured using the accumulated amount of drugs delivered from the pharmacy to ward held stocks. The reliability of this method, particularly the impact of the length of the registration periods, has not been evaluated and such evaluation was aim of the study.
During 26 weeks, we performed a weekly ward stock count of use of broad-spectrum antibiotics - that is second- and third-generation cephalosporins, carbapenems, and quinolones - in five hospital wards and compared the data with corresponding pharmacy sales figures during the same period. Defined daily doses (DDDs) for antibiotics were used as measurement units (WHO ATC/DDD classification). Consumption figures obtained with the two methods for different registration intervals were compared by use of intraclass correlation analysis and Bland-Altman statistics.
Broad-spectrum antibiotics accounted for a quarter to one-fifth of all systemic antibiotics (ATC group J01) used in the hospital and varied between wards, from 12.8 DDDs per 100 bed days in a urological ward to 24.5 DDDs in a pulmonary diseases ward. For the entire study period of 26 weeks, the pharmacy and ward defined daily doses figures for all broad-spectrum antibiotics differed only by 0.2%; however, for single wards deviations varied from -4.3% to 6.9%. The intraclass correlation coefficient, pharmacy versus ward data, increased from 0.78 to 0.94 for parenteral broad-spectrum antibiotics with increasing registration periods (1-4 weeks), whereas the corresponding figures for oral broad-spectrum antibiotics (ciprofloxacin) were from 0.46 to 0.74. For all broad-spectrum antibiotics and for parenteral antibiotics, limits of agreement between the two methods showed, according to Bland-Altman statistics, a deviation of ± 5% or less from average mean DDDs at 3- and 4-weeks registration intervals. Corresponding deviation for oral antibiotics was ± 21% at a 4-weeks interval.
There is a need for caution in interpreting pharmacy sales data aggregated over short registration intervals, especially so for oral formulations. Even a one-month registration period may be too short.
Antibiotic use in hospitals accounts for 10% or less of total antibiotic consumption in most countries, but is characterized by the use of large quantities of broad-spectrum antibiotics (BSAs). Furthermore, hospital departments are densely populated with patients who are at particular risk of acquiring infections of resistant microorganisms [1, 2]. Thus, active surveillance of in-hospital antibiotic use to prevent inappropriate prescribing is a fundamental measure in the struggle against the development of antimicrobial resistance [3, 4].
The most common way of measuring antibiotic use in hospitals is to apply sales data extracted from hospital pharmacy computer systems. However, until now in most European countries pharmacies have not dispensed antibiotics directly to the patients. A ward-held stock of antibiotics has been the routine. In a recent survey of hospital pharmacy practice in Europe it was found that 70% of hospitals hold antibiotic stocks at the wards . When wards purchase drugs from the pharmacy, there will necessarily be a time lag from ordering until consumption. Differences between sales figures and actual drug use might be expected because of variation of stock size over time, discarding of expired drugs, and exchanges of drugs between wards.
Moreover, the duration of the registration intervals may affect the results. In general, short registration intervals have been advocated, especially when the purpose has been to evaluate the effects of specific drug interventions [6–8]. However, the appropriateness of using short registration intervals has not been assessed. How the duration of the registration interval impacts the recording of antibiotic use has not been scrutinized.
Aim of the study was to explore whether the number of defined daily doses (DDDs) reported from the pharmacy, that is to say the sales data to the wards, reflects actual patient consumption of BSAs. In particular, we wanted to evaluate the importance of the length of the registration intervals for the reliability of the pharmacy data.
Oslo University Hospital Aker is a 350-bed tertiary hospital with adult surgical and medical specialities including regional functions for urology, vascular surgery, and endocrinology. In 2007, the number of somatic hospital beds was 356 and 20 060 patients were admitted for 116 251 days of in-patient treatment. Data on occupied bed-days were obtained from the hospital administration, where days of admission and discharge were counted together as one day.
Five hospital wards were included in the study: a pulmonary diseases ward, a combined gastrointestinal and infectious diseases ward, a combined endocrinology and haematology ward, and two urological wards. These wards accounted for 28% of all patient treatment in terms of occupied bed days. The specialties were selected because of a historically relatively high per-ward total use of BSAs, which in our hospital are ciprofloxacin (parenteral and oral) and parenteral formulations of cefuroxime, cefotaxime, ceftazidime, ceftriaxone, meropenem, and imipenem/cilastatin. Ciprofloxacin is the only oral BSA registered in Norway. Piperacillin+tazobactam is the only registered penicillin with an enzyme inhibitor, but this agent was not used in our hospital during the study period.
In Norway, a full assortment of antibiotics is normally stock-piled in the wards and administered by the ward nurses. Antibiotics are not ordered from the pharmacy on a per-patient basis. The ward nurses routinely prepare parenteral antibiotics, retrieved from the ward stock, just prior to administration. On rare occasions, when a drug is out of stock, it will be available from another ward or, in the daytime, from the hospital pharmacy. An emergency pharmacy service exists for essential and rarely used drugs; however, this does not apply for antibacterial agents.
Ward stock accounting
During 26 weeks from October 2006 to April 2007, a pharmacist performed weekly counts of BSA stock in the five wards. The number of vials, infusion bags, and tablets were registered for each ward once a week, before daily orders of antibiotics were placed to the pharmacy, but after morning doses of antibiotics were administered to the patients. The milligram amounts for each antibiotic were converted to defined daily doses units in accordance with the latest WHO ATC/DDD version .
The weekly amount of antibiotics consumed by patients (henceforth designated as "ward BSA") was calculated as the difference between the previous and the current week count, and also taking into account input and output to ward stock. Factors increasing the ward stock size were delivery from the pharmacy and loan from other wards. Factors reducing the ward stock size were loan to other wards, BSAs sent with patients for use after discharge, and discarding of old drugs. Discarded drugs were registered electronically in the pharmacy sales system while the other stock reducing factors were registered manually by ward nurses.
Because of the lack of a ward stock count during holidays in study weeks 10 - 12 and 25 - 26, a weekly average was used for these weeks. Also, two weeks had to be omitted from a total of 26 weeks to establish complete 3- and 4-weeks registration intervals. We chose to omit the weeks 12 and 26, weeks with incomplete data acquisition, thus probably reducing the risk of error.
Pharmacy sales data
For each ward, weekly sales figures for BSA DDDs for the same 26-week period were extracted from the hospital's pharmacy computer system (FarmaPro version 4.1.0a, NAF-data Corp. 2007, Oslo, Norway). In addition to economic data, the system registers the number of DDDs for each antibiotic order placed by the separate wards, as well as total amounts of antibiotics returned by the same wards to the pharmacy provided that shelf-life and condition allow for further use. The resulting sales figures for different time intervals, designated as "pharmacy BSA", were compared with "ward BSA". We chose to assign a weekly average for the study weeks 10 - 12 and 25 - 26 also for the pharmacy sales data (see Discussion). The shortest registration interval was one week, and accumulated periods of two, three, and four weeks were also investigated.
The numbers of DDDs for each BSA and for each ward obtained by the two methods were entered into a Microsoft Excel (version 2007) spreadsheet. All entries and subsequent calculations were double-checked by two of the investigators (JBH and RM).
All statistical analyses were performed using Stata software version 11 (StataCorpLP, College Station, TX USA). The reliability or overall agreement of the pharmacy BSA with ward BSA was assessed by intraclass correlation coefficients (ICC) using a mixed model ANOVA. The theoretical formula for is ICC=σ s 2+σe 2, where σ s 2 is the between-subject variance and σ e 2 is the within-subject variance. The ICC will be high if the measures are in agreement (i.e., the slopes of the regression lines are near 1), and the variation between ward data is large relative to the variation between pharmacy data measurements. A calculated ICC = 1 reflects perfect reliability of the method as evaluated against the assumed "gold standard". An ICC of 0.7 is commonly used as a threshold of sufficient reliability .
Using the method described by Bland and Altman [11, 12], the differences between pairs of DDD measurements for pharmacy BSAs and for ward BSAs were plotted against their averages. Twelve plots (all antibiotics, parenteral antibiotics, and oral antibiotics assessed for one to four weeks registration intervals) were inspected for aberrant trends over the measurement range. Levels of agreement for different combinations were then determined and related to the average of mean DDDs for the two measurements.
Use of broad-spectrum antimicrobial agents
BSA use during the 26 study weeks ranged from 12.8 DDDs/100 bed days in one urological ward to 24.5 DDDs/100 bed days in the pulmonary diseases ward. BSA amounts ranged from 615.3 DDDs at one urological ward to 1144.8 DDDs at the pulmonary diseases ward (Table 1), which represents 19.4% - 26.3% of the total consumption of all systemic anti-bacterial agents (ATC group J01). In the urology and endocrinology/haematology wards, ciprofloxacin was the predominant BSA used. In the pulmonary diseases ward, cephalosporins accounted for two-thirds of total BSA use, whereas the use of the various antibiotics was more evenly distributed in the gastroenterology/infectious diseases ward.
Pharmacy versus ward DDD registrations
During the 26 study weeks there were 1040 dual registrations (pharmacy versus ward data) of the eight BSAs at the five wards. In 550 of these registrations, no use was registered by either of the two methods and this was primarily caused by a very low use of carbapenems (e.g. no use in the urological wards). The number of DDDs that were discarded (10.5), sent with patients on discharge (18.0) or loaned to other wards (14.5) represented 1.1% of the total consumption. Only nine DDDs were borrowed from other wards.
Total BSA use over 26 weeks measured as ward BSA was 3 970 DDDs with a weekly range of 104.7 - 257.2 DDDs. The corresponding total number for pharmacy sales data were almost identical at 3 977 DDDs, with a weekly range of 41.5 - 324.5 DDDs (Figure 1). The largest discrepancy between total ward and pharmacy BSAs was noted in observation weeks 17 - 18 and was related to oral ciprofloxacin. Half of the total BSA use in this study was found to be ciprofloxacin and 77% of this was oral formulation. In both urological wards, the total pharmacy BSA was markedly higher than ward BSA (6.4% - 6.9%). This is in contrast to the three medical wards where the total pharmacy BSA was somewhat lower than ward BSA (1.2% - 4.3%).
The pharmacy sale to the wards per individual order of antibiotics varied little for the various parenteral formulations (median 5 - 10 DDDs) as distinct from oral formulations (ciprofloxacin tablets) for which both larger bulks and a wider range of order size (5 - 50 DDDs, median 20) were registered (Figure 2). The total number of orders varied considerably from 14 (ceftriaxone) to 110 (cefuroxime).
The reliability of pharmacy data as compared to ward stock accounting increased with longer surveillance intervals (Table 2). For total BSAs, all wards included, an intraclass correlation coefficient (ICC) of 0.65 (CI 0.61 - 0.69) was found for one-week intervals whereas for 4-week registration intervals the ICC was 0.90 (CI 0.88 - 0.93). The reliability of pharmacy data was markedly higher for parenteral than for oral BSAs, the latter achieving an ICC of 0.74 (CI 0.58 - 0.90) only at four-week registration, whereas parenteral BSAs had already reached this level at the one-week registration interval.
For all of the 12 combinations of antibiotics (all, parenteral, oral) and periods of registrations (one to four weeks), Bland-Altman statistics revealed mean differences between -0.136 to 0.158 DDDs. The limits of agreement for each of the above combinations were converted to corresponding DDD ranges (Table 3). For all BSAs and parenteral BSAs, limits of agreement of < ± 5% were found for 3- and 4-weeks registration intervals while oral antibiotics (ciprofloxacin) deviated ± 21% from the average mean DDD use even at the 4-week registration interval. Bland-Altman plots (Figure 3) showed diverging differences with increasing averages of DDDs for oral BSAs and for the shorter registration intervals in general; a trend which was far less pronounced for parenteral BSAs and all BSAs at the 4-weeks intervals.
Accurate information regarding antibiotic consumption is a prerequisite for evaluating antibiotic use and implementing measures to avoid excessive prescribing and increased bacterial resistance. To achieve efficient monitoring, short registration periods have been advocated, preferably as short as one month . In this study, we found that surveillance by use of pharmacy sales data was sufficiently reliable for the total registration period of six months. A small mean difference between measurements for all registration intervals, as demonstrated by Bland-Altman statistics, implies that our comparison of pharmacy with ward registrations was not burdened with systematic bias. However, when data for the shorter periods, such as one to four weeks, were investigated, ICCs indicated that pharmacy sales data were not sufficiently reliable, particularly so for the one- to three-week registrations. It will be a clinical decision to define acceptable limits of agreement. Nonetheless, the wide DDD range for oral BSAs (± 21%) even at a four weeks registration interval seems unacceptable by any standards.
The amount of BSAs exchanged between wards, discarded due to exceeded durability or given to patients at discharge was less than 2%, and it was the stock size fluctuations in the wards that accounted for the main discrepancy between pharmacy and ward data. The greatest variations were observed for ciprofloxacin tablets, which largely explain the less reliable pharmacy figures for oral compared to parenteral formulations. This in turn is related to the fact that tablets usually are less voluminous and also cheaper per DDD and may be stored for longer periods than injectable preparations.
Few other studies have evaluated the common method of measuring hospital use of antibiotics by recording pharmacy dispensing or sales data and, to our knowledge, none has previously evaluated the impact of the length of the registration interval. One short report found a poor correlation between a pharmacy dispensing system and an intensive care unit (ICU) based electronic chart as source of data. The investigators speculated that transfer of antibiotics between wards, wastage, and data-entry errors may have been the reasons for the discrepancy . Another study compared pharmacy sales data, based on pharmacy dispensing, with data from nursing records and found that up to 20% of parenteral doses of antibiotics dispensed at the pharmacy were not administered to patients . Such a marked deviation is probably due to delayed transmission of drug information between the ward and the pharmacy, a situation that arises when parenteral antibiotics are prepared at the pharmacy and not near the patient. Neither of the studies evaluated the length of the registration period.
A limitation of our study is the relatively low BSA consumption in our institution, such that the number of registered DDDs was moderate. In hospitals with more extensive BSA consumption and a larger assortment of antibiotic substances, the findings may differ. Also of note, for two periods weekly ward data were missing because of holidays, and for the weeks in question, weekly averages were the basis for the analyses. However, since the corresponding pharmacy sales data were averaged accordingly, this deviation tend to introduce bias towards higher levels of agreement (alpha error), particularly so for the short registration intervals.
We propose that our method of weekly ward stock DDD accounting is an accurate method for indicating actual antibiotic use by patients. Only a much more labour-intensive patient chart review would be more accurate. Although also our method is demanding and therefore not practical for routine surveillance, we regard it as well suited for scientific purposes and for quality assessments of pharmacy sales figures. In some hospitals, electronic patient charts have been introduced and this allows for another method, probably with a high level of accuracy, for measuring drug consumption . However, for the immediate future in most European hospitals, the pharmacy will remain the principal data source for surveillance of use of antibiotics.
Pharmacy sales data for total BSA use were representative for the actual drug consumption when longer registration periods were used. For the data to be sufficiently reliable, a four-week registration period is required for parenteral formulations, whereas for oral medications one month is not sufficient. For analyses of BSA subclasses and separate hospital units, even these intervals are probably too short, at least in hospitals with a low-to-moderate consumption profile.
MacKenzie FM, Bruce J, Struelens MJ, Goossens H, Mollison J, Gould IM, on behalf of the ARPAC Steering Group: Antimicrobial drug use and infection control practices associated with the prevalence of methicillin-resistant Staphylococcus aureus in European hospitals. Clin Microbiol Infect. 2007, 13: 269-276. 10.1111/j.1469-0691.2006.01592.x.
Zoutman DE, Ford BD: The relationship between hospital infection surveillance and control activities and antibiotic-resistant pathogen rates. Am J Infect Control. 2005, 33: 1-5. 10.1016/j.ajic.2004.09.003.
Monnet DL: Measuring antimicrobial use: the way forward.[comment]. Clin Infect Dis. 2007, 44: 671-673. 10.1086/511649.
Dellit TH, Owens RC, McGowan JE, Gerding DN, Weinstein RA, Burke JP, Huskins WC, Paterson DL, Fishman NO, Carpenter CF, Brennan PJ, Billeter M, Hooton TM: Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America guidelines for developing an institutional program to enhance antimicrobial stewardship. Clin Infect Dis. 2007, 44: 159-177. 10.1086/510393.
MacKenzie FM, Gould IM, Bruce J, Mollison J, Monnet DL, Krcmery V, Cookson B, van der Meer JW: The role of microbiology and pharmacy departments in the stewardship of antibiotic prescribing in European hospitals. J Hosp Infect. 2007, 65 (Suppl 2): 73-81.
Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, Davey P: Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J Antimicrob Chemother. 2003, 52: 842-848. 10.1093/jac/dkg459.
Kritsotakis EI, Gikas A: Surveillance of antibiotic use in hospitals: methods, trends and targets.[see comment]. Clin Microbiol Infect. 2006, 12: 701-704.
Monnet DL, López-Lozano JM, Campillos P, Burgos A, Yagüe A, Gonzalo N: Making sense of antimicrobial use and resistance surveillance data: application of ARIMA and transfer function models. Clin Microbiol Infect. 2001, 7 (Suppl 5): 29-36.
WHO Collaborating Centre for Drug Statistics Methodology: Guidelines for ATC classification and DDD assignment 2007. WHO Collab Centre, Oslo, Norway. 2007
Hripcsak G, Heitjan DF: Measuring agreement in medical informatics reliability studies. J Biomed Inform. 2002, 35: 99-110. 10.1016/S1532-0464(02)00500-2.
Altman JM, Bland DG: Statistical methods for assessing agreement between two methods of clinical measurements. Lancet. 1986, i: 307-310.
Bland DG, Altman JM: A note on the use of the intraclass correlation coefficient in the evaluation of agreement between two measurement methods. Comput Biol Med. 1990, 20: 337-340. 10.1016/0010-4825(90)90013-F.
Kritsotakis EI, Assithianakis P, Kanellos P, Tzagarakis N, Ioannides MC, Gikas A: Surveillance of monthly antimicrobial consumption rates stratified by patient-care area: a tool for triggering and targeting antibiotic policy changes in the hospital. J Chemother. 2006, 18: 394-401.
Solano T, Matthews Z: Comparison of antibiotic dispensing data with administration data in intensive care. J Hosp Infect. 2004, 58: 87-88. 10.1016/j.jhin.2004.05.010.
Itokazu GS, Glowacki RC, Schwartz DN, Wisniewski MF, Rydman RJ, Weinstein RA: Antimicrobial consumption data from pharmacy and nursing records: how good are they?. Infect Control Hosp Epidemiol. 2005, 26: 395-400. 10.1086/502557.
Polk RE, Fox C, Mahoney A, Letcavage J, MacDougall C: Measurement of adult antibacterial drug use in 130 US hospitals: comparison of defined daily dose and days of therapy. Clin Infect Dis. 2007, 44: 664-670. 10.1086/511640.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/11/166/prepub
Acknowledgements and Funding
This research was performed without funding from any external source. The authors wish to thank Lien My Diep, senior statistician at the Research Centre of Oslo University Hospital, for assistance in conducting the statistical analyses.
The authors declare that they have no competing interests.
JBH designed the study, analyzed and interpreted the data and drafted the manuscript. RM contributed to study design, acquired the ward stock data and helped in interpretation and writing of the manuscript. ÅR advised on design and interpretation of the study, and contributed significantly with critical revisions of the manuscript.
All authors have read and approved of the final manuscript.
About this article
Cite this article
Haug, J.B., Myhr, R. & Reikvam, Å. Pharmacy sales data versus ward stock accounting for the surveillance of broad-spectrum antibiotic use in hospitals. BMC Med Res Methodol 11, 166 (2011). https://doi.org/10.1186/1471-2288-11-166