Comparison of two alternative study designs in assessment of medicines utilisation in neonates
- Georgi Nellis†1, 2Email author,
- Irja Lutsar†1,
- Heili Varendi†2,
- Karolin Toompere†3,
- Mark A Turner4, 5,
- Jennifer Duncan†6 and
- Tuuli Metsvaht†7
© Nellis et al.; licensee BioMed Central Ltd. 2014
Received: 24 July 2013
Accepted: 9 July 2014
Published: 16 July 2014
Estimates of prevalence are known to be affected by the design of cross-sectional studies. A pan-European study provided an opportunity to compare the effect of two cross-sectional study designs on estimates of medicines use.
A Service evaluation survey (SES) and a web-based point-prevalence study (PPS) were conducted as part of a European study of neonatal exposure to excipients. Neonatal units from all European Union countries plus Iceland, Norway, Switzerland and Serbia were invited to participate. All medicines prescribed to neonates were recorded during three-day and one-day study periods in the SES and PPS, respectively. In the PPS individual demographic and prescription data were also collected.
To compare the probabilities that a particular medicine would be reported by each study multilevel mixed effects logistic regression models with crossed random effects were applied. The relationship between medicines exposure at the unit and individual levels in the PPS data was assessed using polynomial regression with square root transformation.
Of 31 invited countries 20 and 21 with 115 and 89 units joined the SES and PPS, respectively. Out of 5,572,859 live births in invited countries in 2010 a higher proportion was covered by units participating in the SES compared to the PPS (11% vs 6%, respectively; OR 1.89; 95% CI 1.87-1.89). A greater number of active pharmaceutical ingredients (API), manufacturers and trade names were registered in the SES compared to the PPS. High correlation between the two studies in frequency of use for each specified API was seen (R2 = 0.86). The average probability of a department to use a given API was greater in the SES compared to the PPS (OR 2.36; 95% CI 2.05-2.73) with higher frequency of use and longer average duration of prescription further increasing the difference. The polynomial regression model described the correlation between APIs exposure on unit and individual level well (R2 = 0.93).
The simple data structure and longer study period of the SES resulted in improved recruitment and higher likelihood of capture for a given API. The frequency of use at the unit level appears a good surrogate of individual exposure rates.
KeywordsPharmacoepidemiologic methods Cross-sectional studies Data collection Drug/excipient exposure
Newborn babies receive a range of medicines many of which have not been studied adequately. Furthermore, exposure to medicines introduces risks and risk assessment requires information about the extent of medicines use. One neglected source of risk is the inclusion of excipients in medicines. Thus there is a need to gather information about medicines use in a way that captures data about active pharmaceutical ingredients (API) and the excipients. Variability in excipient content between formulations of the same API needs to be considered.
Cross-sectional studies are attractive for studies of medicines use [1–7]. In principle, data can be collected at unit or individual level. Unit level data involve listing all the products used on a unit over a specified period of time. These data are simple to collect and can be integrated with processes for medicines supply (such as sales or dispensing activity). Unit level data indicate which medicines are used (including how many distinct products, APIs and excipients are used) and will indicate whether there are geographical differences in medicines use. In combination with demographic data about the units this method can provide indicative estimates of market size. Individual level data require an extra step of data collection. This approach captures denominator data and generates more precise estimates of market size while allowing for stratification according to important clinical variables such as age of the baby. Point prevalence studies (PPS) have been used in antibiotic consumption studies [4, 5, 8]. Their extremely short duration may underestimate exposure to less frequently used medicines. Unit level studies can have a longer duration but may not be manageable in a multinational setting . Although a recent analysis has described the impact of cross-sectional study designs on disease incidence/prevalence estimates , little is known about the effects of study design in research about medicines use. We are not aware of any head to head comparison of different methods and thus the question how to navigate between different methods in medicine exposure research remains unanswered.
Within the European Study of Neonatal Excipient Exposure (ESNEE) we planned to use both unit and individual level data collection in order to maximise yield for two different study questions. . This gave us the opportunity to compare the two study designs – a short duration PPS and longer lasting service evaluation survey (SES). We used APIs as the unit of analysis in this methodological study.
Accordingly, we aimed (1) to describe the implementation of each method; (2) to assess the extent of discordance between the methods by comparing the probabilities that particular medicines would be reported by each method; (3) to explore the correlations between medicines exposure at unit and individual levels.
The SES and PPS were performed as multicentre observational surveys of routine clinical practice. During the SES all medicines prescribed to neonates (≤28 days of postnatal age) were recorded on pre-formulated Excel spreadsheets over three consecutive days within a fixed period of time from May 30th to September 30th, 2011. PPS data collection was performed in a web-based database within one day during one of three fixed two-week study periods from January to February; March or May to June, 2012. Each participating unit was free to choose the most appropriate day(s) for data collection. In both surveys printable data collection forms were provided as an alternative to electronic data insertion.
Participating units and study population
The study aimed to cover all 27 European Union countries plus Iceland, Norway, Switzerland and Serbia. A network of national contact persons (Lead Contacts) was built by the ESNEE consortium. Search for national Lead Contacts was carried out through national professional/scientific societies of neonatology, paediatrics and/or perinatology as well as through personal contacts of the ESNEE consortium members or other FP-7-funded consortia (NeoMero, TINN) [12, 13]. The Lead Contact was then asked to recruit as many hospitals and units providing neonatal care in the country as possible.
At the outset, a PPS sample size estimation was planned. It was intended to base this on cluster sampling analysis stratified by country, assuming conservatively a 15% neonatal admission rate of all live births [14–17] and a response rate of 50-70% of invited units as described in previous neonatal surveys [18–21]. Based on the Eurostat NUTS (or equivalent if NUTS classification not available for the country) regional distribution of the population and reported nation-wide birth rates, the number of potentially available neonatal admissions would be calculated and the representative sample size estimated for each country and region . A complete list of institutions involved in neonatal care would be created for each randomly chosen NUTS2 region with subsequent involvement of all units to cover a whole region with proportional representation of different unit levels. Alternatively, if a full list of neonatal units was not available and/or the response rate in the SES would suggest critically low recruitment, all contacts provided by Lead Contacts were to be invited to participate in the PPS.
All general neonatal, intermediate and neonatal intensive care (NICU) as well as mixed paediatric and neonatal intensive care units with more than 50% of admissions consisting of neonates were eligible with stratification according to the level of care: level 1 with special neonatal care; level 2 with high dependency care, short term intensive care and low birth weight care, and level 3 with comprehensive intensive care for extremely low birth weight infants available [23, 24]. Departments, offering different levels of care were classified according to the highest level they provided.
In both studies all eligible neonates in the unit at 8 a.m. of the study day and in the SES additionally those admitted within the next 72 h, were included. The SES recorded the number of neonates receiving any prescription during the study period. In the PPS individual demographic data including gender, gestational age (GA), birth weight, 1st and 5th minute Apgar score, current body weight, postnatal age and organ dysfunctions, were recorded for all neonates receiving any prescriptions active on the study morning at 8 a.m.
All prescriptions, including micronutrients, iron, vitamins, parenteral nutrition solutions and topical agents were recorded. Blood products, glucose and electrolyte solutions, vaccines, nursery care topical agents, herbal medicines and food including breast milk fortifiers were excluded. The following information was collected for every medicine prescribed: trade name, manufacturer, API, strength, pharmaceutical dosage form and route of administration. Additionally individual dosing regimen together with prescription starting date was collected in the PPS.
Extemporaneous or compounded forms of medicines were not included in described analysis. The use of caffeine and morphine was reported differently in the two studies. UK units using ´special´ manufactured  preparations, rather than extemporaneous formulations used in the rest of European countries, were over-represented in the PPS. Accordingly, these two APIs were excluded from the analysis.
In both studies the data were pooled for analysis without any comment on treatment strategies of individual participating units.
Data management and statistical analysis
SES data collection forms were checked for legibility with two rounds of queries regarding missing data (response rate 52%). Each medication was classified according to brand name, manufacturer and pharmaceutical dosage form. The list of medicines obtained in the SES was used to prepopulate choices in the PPS database. To minimize the risk of data entry errors all ´newly appearing´ medicines in the PPS were added to the database by the ESNEE team.
Statistical analysis was performed with Stata Software (ver. 12.1). Descriptive statistics were used as appropriate to describe the variability of medicine exposure in both studies.
To identify the effect size of the study method, the average probabilities of the units to use any given API in SES vs PPS were compared by multilevel mixed effects logistic regression models with crossed random effects . Since the sample of departments in the two studies was partly overlapping, department and API were added to the model as a random effect when comparing the average odds of using an API between the two study methods. The outcome variable was a binary indicator showing whether an API was used in a specific unit in a specific study. The analysis was limited to agents used in more than one unit in the SES. All models were adjusted for geographic region (Northern, Southern, Western and Eastern Europe according to the United Nations Statistics department ) and unit level. These variables were identified as potential confounders in an exploratory population averaged Poisson regression model with robust standard errors  analysis including 10 most often used APIs. Furthermore the effect of unit size (described by number of annual admissions), frequency of a given medicine use (described by proportion of units using it) and the duration of the prescription (described by the average time from the start of prescription to the PPS study day) on the estimated method effect size was explored.
To study the relationship between the number of units using each API and the number of prescriptions for that API polynomial regression model with square root transformation was used based on the PPS data.
The SES did not collect any personal data and so did not require ethics committee approval in any of the participating countries. For participation in the PPS Ethics Committee approval was obtained in compliance with the respective national guidelines. In compliance with local authorities no consent for individual patients was sought, as all data were collected in routine clinical practice and anonymised before leaving study sites.
Characterisation of participating units
Characteristics of participating units in the SES and the PPS
OR (95% CI)
No of participating countries
No of participating hospitals
No of participating units
Median (IQR) No of participating units per country
No (%) of teaching hospitals
Level distribution (%)
Total annual live births in catchment area of participating units* (year of reference: 2010)
Regional proportions of live births covered (%)**
Annual number of admissions in participating units α
Annual admissions coverage; mean% (SD) ¥
No of neonates in unit in the study period
No of neonates receiving any drug prescription during the study period
Consumption of medicines in the PPS and the SES
Medicines consumption in the SES and the PPS
OR (95% CI)
No of prescriptions during the study period
Median (IQR) No of prescriptions per neonate
No of active ingredients prescribed*
No of trade names (by name, manufacturer, pharmaceutical dosage form and strength)
No of manufacturers
Route of administration (%)
Correlation of medicine use between SES and PPS
In multilevel mixed effects logistic regression models with crossed random effects the average probability of the departments to use each of the most common API was higher in the SES compared to the PPS (OR 2.36; 95% CI 2.05-2.73; p < 0.0005). The frequency of use and average duration of prescription further increased the likelihood of being registered in favor of the SES by 1.01 times (95% CI 1.01-1.02; p < 0.0005) and 1.02 times (95% CI 1.00-1.05; p = 0.047) per each additional percent in frequency and day in duration of prescription, respectively. Size of the department did not influence the capture probability (OR 0.99; 95% CI 0.99-1.00; p = 0.652).
By applying two different study designs in comparable target populations we have demonstrated that simple data structure and longer study period used in the SES improved recruitment and the likelihood of capture of medicines consumption. The probability of capture for a given API was further enhanced by higher frequency of use and longer duration of prescription. This meant that the SES gave a more comprehensive list of trade names, APIs and manufacturers than the PPS. The list of most frequently used APIs was overlapping in 90% with high correlation of use frequencies between the two studies.
The strength of both methods described here is the simplicity and uniformity of data collection. A two-step approach, as used in the ESNEE project may allow simultaneous coverage of both unit-level and individual-level data in neonatal excipient exposure studies, where hardly any multi-country data are available. We have recruited so far one of the largest international cohorts of neonatal units from 21 European countries revealing a surprisingly wide list of medicines. The number of trade names applied in NICU setting was also high. The response rate of 71% on country level as well as the number of prescriptions per neonate [9, 29] are similar to those reported previously [18–21]. Against the background of rising safety concerns of neonatal medicine/excipient exposure, the wide trade name list from the SES allows identification of substitution possibilities, while individual exposure data from the PPS provide a more precise quantification of the problem.
Different study designs can be applied in pharmacoepidemiology [30, 31]. Cohort studies have the advantage of allowing data collection over prolonged time periods which is more likely to capture rare exposures. However, the expense and duration make it hard to implement cohort designs in multinational studies . Case–control studies allow rich data collection when a limited number of medicines/excipients are targeted and data collection is limited to a few centers. The PPS and SES, in contrast, are easy to perform in multinational settings. They are time and resource saving and to some extent cover each other`s shortcomings like under- or overestimated exposure rates of rarely used medicines or need for individual data . Alternately, nested study designs with longitudinal data collection allow linkage between different datasets and may prove the best option. However, to the best of our knowledge, it has hardly been used in paediatric pharmacoepidemiological studies.
When planning medicine/excipient exposure studies the research question to be answered needs to be balanced against the implications of study design. We recognize that a 3-day survey is actually very short in duration. However, even this small longitudinal component resulted in a more comprehensive list of medicines due to the better recruitment when simple data structure was used. On the other hand, when individual exposures of frequently used medicines are targeted, the increasing data volume (both structure and duration) needs to be weighed against evolving decrease in compliance that may lead to underestimation of prevalence . It has been shown that the longer the recall period, the greater the imprecision (especially underestimation) of prevalence estimates [33, 34], although this finding may be more relevant in non-medical informants . Furthermore, not only optimal recruitment but also sample size maintenance over the course of a study becomes an issue .
Additionally, economic evaluation plays a role in the choice of a specific study design. There is considerable overlap between acceptable methodologies and those sanctioned by health economists . Although pharmacoeconomic analysis remains beyond the scope of this study, a few comments can be made. There were no major cost-differences in conducting these studies with the exception of those associated with the development of the PPS database. Web-based collection with automated control of data completeness was applied to improve compliance and quality of reporting. We believe that wherever sufficient to answer the research question, simple data structure is favourable in terms of resource allocation. However, when a more complex format is required, reasonable additional expenditure on data collection tools may result in an improved cost-benefit ratio.
We have also shown a high correlation between APIs exposure on unit and individual level. This should allow an indirect individual exposure assessment from less demanding approach like SES under the condition of limited resources. We realize, that the estimates in the frequency of medicine use in the PPS are systematically lower compared to the SES. However, knowing the relationship between the two an estimate for one can be extrapolated from the other. Nevertheless, such calculations should be applied with caution and to estimate the exposure only for the studied population as a whole. In neonatal studies stratification by GA age allows a more meaningful risk assessment due to the strong dependence of the maturity of metabolic pathways on the postmenstrual age of the newborn [37, 38]. It has been shown previously and also by us that medicines utilization pattern in extreme prematurity is different from that near or at term [9, 29, 39, 40]. Therefore, the PPS approach should be preferred to obtain individual exposure estimates in neonates of different GA.
Some limitations of the present study need to be noted. First, the quality of data collection was dependent on the personnel entering the data. In both studies data insertion was done by medical specialists according to standardized protocols. As for the quality of reported data, in the SES most queries concerned demographic data; only a few queries were related to the information about medicines. Second, neonatal units were given the option of selecting the most appropriate day(s) for data collection. This could lead to underestimation of medicine use because less busy day(s) probably were more likely to be chosen. However, we believe that eventually this approach captured information on a larger number of neonates and medicines through improved compliance and a larger number of participating units. Third, the slightly different representation by country required exclusion of two APIs from final analyses due to differences in regional handling/approach. Fourth, the SES and the PPS were done at different times. Nested design with detailed individual data collection (PPS) performed as part of the SES was not feasible, as an interim between the two studies was needed to prepopulate the PPS database from the SES list of medicines. This could potentially reduce the overlap between the studies. In that situation our data about overlap represent a “worst case scenario”. The true concordance in API reporting may be higher than described by us. Finally, about two thirds of the 31 invited countries agreed to participate with relative inhomogeneity of the regional distribution of participating units. Although the inability to randomly select hospitals probably caused overrepresentation of teaching hospitals (61% and 77% in the SES and PPS, respectively, compared to less than 10%, reported in national hospital statistics ), data on the distribution of the levels of neonatal care will allow appropriate adjustments. Both SES and PPS have achieved representative sample size on patient level, the margin of error being 1.73% and 2.63%, respectively (CI 95%; response distribution 50%; population size 5.4 million live births). However, it would be valid under the condition of randomisation that remained unfeasible in our studies. Still, the current analysis gives a notion of good coverage of the target population in the SES and PPS.
The less demanding approach of the SES allows better recruitment of units for longer periods resulting in a more comprehensive list of medicines and trade names. As we have shown, the frequency of use on unit level appears a good surrogate of individual exposure rates. However, more detailed information can be collected in a PPS. Investigators need to balance the advantages of a PPS with the risk of bias, which depends on the frequency as well as duration of medicine use. Simultaneous use of both methods with merged data analysis will likely result in optimal coverage of both aspects of the problem.
Active Pharmaceutical Ingredient
European Study of Neonatal Excipient Exposure
Neonatal Intensive Care Unit
Nomenclature of Territorial Units for Statistics
Point Prevalence Study
Service Evaluation Survey.
We thank all national contact persons who provided lists of neonatal units in their countries: Bernhard Resch (Austria); Pieter De Cock (Belgium); Nelly Jekova (Bulgaria); Elisabeth Iyore (Denmark); Pascal Vaconsin (France); Kosmas Sarafidis (Greece); Aranka Vegso (Hungary); Noreen O'Callaghan (Ireland); Rocco Agostino (Italy); Daiga Kviluna (Latvia); Rasa Tameliene (Lithuania); Rene F. Kornelisse (Netherlands); Dag Bratlid (Norway); Almerinda Pereira (Portugal); Maria Livia Ognean (Romania); Milica Bajcetic (Serbia); Darja Paro (Slovenia); Elizabeth Valls (Spain); Per Nydert (Sweden); Hans Ulrich Bucher (Switzerland); Maria Cordina (Malta).
We also thank the whole ESNEE project consortium for support: S. Graham, A. Rieutord, T. Storme, P. Vaconsin, A.J. Nunn, J. McElnay, J. Millership, S. Yakkundi, H. Mulla, H. Pandya, J.Lass.
ESNEE is funded through ERA-NET PRIOMEDCHILD by the following national agencies: MRC from United Kingdom, Estonian Science Foundation from Estonia, Agence Nationale de la Recherche from France.
- Roberts JA, De Waele JJ, Dimopoulos G, Koulenti D, Martin C, Montravers P, Rello J, Rhodes A, Starr T, Wallis SC, Lipman J: DALI: Defining Antibiotic Levels in Intensive care unit patients: a multi-centre point of prevalence study to determine whether contemporary antibiotic dosing for critically ill patients is therapeutic. BMC Infect Dis. 2012, 12: 152-View ArticlePubMedPubMed CentralGoogle Scholar
- Zarb P, Amadeo B, Muller A, Drapier N, Vankerckhoven V, Davey P, Goossens H: Antifungal therapy in European hospitals: data from the ESAC point-prevalence surveys 2008 and 2009. Clin Microbiol Infect Off Publ Eur Soc Clin Microbiol Infect Dis. 2012, 18: E389-E395.Google Scholar
- Ingram PR, Seet JM, Budgeon CA, Murray R: Point-prevalence study of inappropriate antibiotic use at a tertiary Australian hospital. Intern Med J. 2012, 42: 719-721.View ArticlePubMedGoogle Scholar
- Amadeo B, Zarb P, Muller A, Drapier N, Vankerckhoven V, Rogues A-M, Davey P, Goossens H: European Surveillance of Antibiotic Consumption (ESAC) point prevalence survey 2008: paediatric antimicrobial prescribing in 32 hospitals of 21 European countries. J Antimicrob Chemother. 2010, 65: 2247-2252.View ArticlePubMedGoogle Scholar
- Ansari F, Erntell M, Goossens H, Davey P: The European surveillance of antimicrobial consumption (ESAC) point-prevalence survey of antibacterial use in 20 European hospitals in 2006. Clin Infect Dis an Off Publ Infect Dis Soc Am. 2009, 49: 1496-1504.View ArticleGoogle Scholar
- Zarb P, Amadeo B, Muller A, Drapier N, Vankerckhoven V, Davey P, Goossens H: Identification of targets for quality improvement in antimicrobial prescribing: the web-based ESAC Point Prevalence Survey 2009. J Antimicrob Chemother. 2011, 66: 443-449.View ArticlePubMedGoogle Scholar
- Sequi M, Campi R, Clavenna A, Bonati M: Methods in pharmacoepidemiology: a review of statistical analyses and data reporting in pediatric drug utilization studies. Eur J Clin Pharmacol. 2013, 69: 599-604.View ArticlePubMedGoogle Scholar
- Zarb P, Goossens H: European Surveillance of Antimicrobial Consumption (ESAC). Drugs. 2011, 71: 745-755.View ArticlePubMedGoogle Scholar
- Lass J, Käär R, Jõgi K, Varendi H, Metsvaht T, Lutsar I: Drug utilisation pattern and off-label use of medicines in Estonian neonatal units. Eur J Clin Pharmacol. 2011, 67: 1263-1271.View ArticlePubMedGoogle Scholar
- Schmidt W-P, Arnold BF, Boisson S, Genser B, Luby SP, Barreto ML, Clasen T, Cairncross S: Epidemiological methods in diarrhoea studies–an update. Int J Epidemiol. 2011, 40: 1678-1692.View ArticlePubMedPubMed CentralGoogle Scholar
- Turner M: European Study of Neonatal Exposure to Excipients (ESNEE). Infant. 2011, 7: 1-4.Google Scholar
- NEOMERO - European multicenter network to evaluate pharmacokinetics, safety and efficacy of Meropenem in neonatal sepsis and meningitis. http://www.neomero.org/,
- TINN European Survey on the Use of Antibiotics in NICUs. http://www.tinn-project.org/,
- Labenne M, Michaut F, Gouyon B, Ferdynus C, Gouyon J-B: A population-based observational study of restrictive guidelines for antibiotic therapy in early-onset neonatal infections. Pediatr Infect Dis J. 2007, 26: 593-599.View ArticlePubMedGoogle Scholar
- Dani C, Corsini I, Piergentili L, Bertini G, Pratesi S, Rubaltelli FF: Neonatal morbidity in late preterm and term infants in the nursery of a tertiary hospital. Acta Paediatr (Oslo, Norw 1992). 2009, 98: 1841-1843.View ArticleGoogle Scholar
- Tracy SK, Tracy MB, Sullivan E: Admission of term infants to neonatal intensive care: a population-based study. Birth. 2007, 34: 301-307.View ArticlePubMedGoogle Scholar
- Schiariti V, Klassen AF, Houbé JS, Synnes A, Lisonkova S, Lee SK: Perinatal characteristics and parents’ perspective of health status of NICU graduates born at term. J Perinatol Off J Calif Perinat Assoc. 2008, 28: 368-376.View ArticleGoogle Scholar
- Losacco V, Cuttini M, Greisen G, Haumont D, Pallás-Alonso CR, Pierrat V, Warren I, Smit BJ, Westrup B, Sizun J: Heel blood sampling in European neonatal intensive care units: compliance with pain management guidelines. Arch Dis Child Fetal Neonatal Ed. 2011, 96: F65-F68.PubMedGoogle Scholar
- Zeitlin J, Papiernik E, Bréart G: Regionalization of perinatal care in Europe. Semin Neonatol SN. 2004, 9: 99-110.View ArticlePubMedGoogle Scholar
- Greisen G, Mirante N, Haumont D, Pierrat V, Pallás-Alonso CR, Warren I, Smit BJ, Westrup B, Sizun J, Maraschini A, Cuttini M: Parents, siblings and grandparents in the Neonatal Intensive Care Unit. A survey of policies in eight European countries. Acta Paediatr (Oslo, Norw 1992). 2009, 98: 1744-1750.View ArticleGoogle Scholar
- Cuttini M, Rebagliato M, Bortoli P, Hansen G, de Leeuw R, Lenoir S, Persson J, Reid M, Schroell M, de Vonderweid U, Kaminski M, Lenard H, Orzalesi M, Saracci R: Parental visiting, communication, and participation in ethical decisions: a comparison of neonatal unit policies in Europe. Arch Dis Child Fetal Neonatal Ed. 1999, 81: F84-F91.View ArticlePubMedPubMed CentralGoogle Scholar
- NUTS - Nomenclature of territorial units for statistics. [http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction]
- Stark AR: Levels of neonatal care. Pediatrics. 2004, 114: 1341-1347.View ArticlePubMedGoogle Scholar
- Modi N: Categories of Care 2011. British association of perinatal medicine. 2011, London, UKGoogle Scholar
- Manufacturer’s and wholesale dealer's licences. http://www.mhra.gov.uk/Howweregulate/Medicines/Licensingofmedicines/Manufacturersandwholesaledealerslicences/index.htm,
- Raudenbush SW: A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research. J Educ Stat. 1993, 18: 321-349.View ArticleGoogle Scholar
- United Nations Statistics Division - composition of macro geographical (continental) regions, geographical sub-regions, and selected economic and other groupings. http://unstats.un.org/unsd/methods/m49/m49regin.htm#europe,
- Zou GY, Donner A: Extension of the modified Poisson regression model to prospective studies with correlated binary data. Stat Methods Med Res. 2013, 22: 661-670.View ArticlePubMedGoogle Scholar
- Neubert A, Lukas K, Leis T, Dormann H, Brune K, Rascher W: Drug utilisation on a preterm and neonatal intensive care unit in Germany: a prospective, cohort-based analysis. Eur J Clin Pharmacol. 2010, 66: 87-95.View ArticlePubMedGoogle Scholar
- Turner MA, Duncan J, Shah U, Metsvaht T, Varendi H, Nellis G, Lutsar I, Yakkundi S, McElnay J, Pandya H, Mulla H, Vaconsin P, Storme T, Rieutord A, Nunn AJ: Risk assessment of neonatal excipient exposure: Lessons from food safety and other areas. Adv Drug Deliv Rev. 2014, 73C: 89-101.View ArticleGoogle Scholar
- Verhamme K, Sturkenboom M: Study designs in paediatric pharmacoepidemiology. Eur J Clin Pharmacol. 2011, 67 (Suppl 1): 67-74.View ArticlePubMedGoogle Scholar
- White HL, Macpherson AK: Capturing paediatric injury in Ontario: differences in injury incidence using self-reported survey and health service utilisation data. Inj Prev. 2012, 18: 33-37.View ArticlePubMedGoogle Scholar
- Melo MCND, Taddei JAD AC, Diniz-Santos DR, May DS, Carneiro NB, Silva LR: Incidence of diarrhea: poor parental recall ability. Brazilian J Infect Dis an Off Publ Brazilian Soc Infect Dis. 2007, 11: 571-579.View ArticleGoogle Scholar
- Alam N, Henry FJ, Rahaman M: Reporting errors in one-week diarrhoea recall surveys: experience from a prospective study in rural Bangladesh. Int J Epidemiol. 1989, 18: 697-700.View ArticlePubMedGoogle Scholar
- Knottnerus JA, Tugwell P: Ongoing progress in research on drug exposure and adverse events. J Clin Epidemiol. 2012, 65: 1029-1030.View ArticlePubMedGoogle Scholar
- Gagnon JP, Rindress D: Pharmacoeconomics: Identifying The Issues. 1998, USA: International Society for Pharmacoeconomics and Outcomes ResearchGoogle Scholar
- LeBel M, Ferron L, Masson M, Pichette J, Carrier C: Benzyl alcohol metabolism and elimination in neonates. Dev Pharmacol Ther. 1988, 11: 347-356.PubMedGoogle Scholar
- Hiller JL, Benda GI, Rahatzad M, Allen JR, David H, Carlson CV, Reynolds JW, Culver DH, Carison CV: Benzyl alcohol toxicity: impact on mortality and intraventricular hemorrhage among very low birth weight infants the online version of this article, along with updated information and services, is located on the world wide web at: benzyl alcohol toxic. Pediatrics. 1986, 77: 500-506.PubMedGoogle Scholar
- Warrier I, Du W, Natarajan G, Salari V, Aranda J: Patterns of drug utilization in a neonatal intensive care unit. J Clin Pharmacol. 2006, 46: 449-455.View ArticlePubMedGoogle Scholar
- Clark RH, Bloom BT, Spitzer AR, Gerstmann DR: Reported medication use in the neonatal intensive care unit: data from a large national data set. Pediatrics. 2006, 117: 1979-1987.View ArticlePubMedGoogle Scholar
- Zarb P, Coignard B, Griskeviciene J, Muller A, Vankerckhoven V, Weist K, Goossens M, Vaerenberg S, Hopkins S, Catry B, Monnet D, Goossens H, Suetens C: The European Centre for Disease Prevention and Control (ECDC) pilot point prevalence survey of healthcare-associated infections and antimicrobial use. Euro Surveill Bull Eur sur les Mal Transm = Eur Commun Dis Bull. 2012, 17 (46): pii=20316Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2288/14/89/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.