Skip to main content
  • Research article
  • Open access
  • Published:

A snapshot of pneumonia research activity and collaboration patterns (2001–2015): a global bibliometric analysis



This article describes a bibliometric review of the scientific production, geographical distribution, collaboration, impact, and subject area focus of pneumonia research indexed on the Web of Science over a 15-year period.


We searched the Web of Science database using the Medical Subject Heading (MeSH) of “Pneumonia” from January 1, 2001 to December 31, 2015. The only document types we studied were original articles and reviews, analyzing descriptive indicators by five-year periods and the scientific production by country, adjusting for population, economic, and research-related parameters.


A total of 22,694 references were retrieved. The number of publications increased steadily over time, from 981 publications in 2001 to 1977 in 2015 (R2 = 0.956). The most productive country was the USA (38.49%), followed by the UK (7.18%) and Japan (5.46%). Research production from China increased by more than 1000%. By geographical area, North America (42.08%) and Europe (40.79%) were most dominant. Scientific production in low- and middle-income countries more than tripled, although their overall contribution to the field remained limited (< 15%).

Overall, 18.8% of papers were the result of an international collaboration, although this proportion was much higher in sub-Saharan Africa (46.08%) and South Asia (23.43%). According to the specific MeSH terms used, articles focused mainly on “Pneumonia, Bacterial” (19.99%), followed by “Pneumonia, Pneumococcal” (7.02%) and “Pneumonia, Ventilator-Associated” (6.79%).


Pneumonia research increased steadily over the 15-year study period, with Europe and North America leading scientific production. About a fifth of all papers reflected international collaborations, and these were most evident in papers from sub-Saharan Africa and South Asia.

Peer Review reports


Pneumonia is an important infectious disease worldwide and is associated with high morbidity, mortality and health system expenditure [1, 2]. In 2015, data from the Global Burden of Disease study showed that lower respiratory tract infections, including pneumonia, were the third most common cause of death, exceeded only by ischemic heart disease and cerebrovascular disease [3]. Community-acquired pneumonia (CAP) remains the primary cause of death from infectious disease globally, and its high impact on morbidity and mortality is especially concentrated in children under five and the elderly [1, 4,5,6]. The World Health Organization (WHO) predicted that deaths from lower respiratory tract infections would remain among the top four causes of deaths up to at least 2030 [7]. Antibiotic-resistant strains have also been on the rise, although resistance does not appear to be related to mortality. However, pneumonia is associated with high rates of hospitalization and length of hospital stay. Moreover, it has considerable long-term effects on quality of life, and long-term prognosis is worse in patients with pneumococcal pneumonia [1].

Despite the public health importance of the disease, few studies have evaluated research in the area using bibliometric methods. Indeed, only Head et al. (2015) have analyzed publications on pneumonia, and their work was limited in geographical scope to the UK [8, 9]. In this study, by analyzing scientific papers on pneumonia published in the main international scientific journals, we aimed to identify the scientific contribution of different countries to the worldwide research effort, the most cited landmark articles, the degree and nature of scientific collaboration, and the topics addressed.

This bibliometric description can provide relevant information for researchers in the field, particularly new scientists, giving a snapshot of strong research areas in pneumonia and global health as well as possible gaps requiring additional investments [10,11,12]. The paper also provides clues for addressing the weaknesses observed, such as the need to promote North-South collaborations and other research initiatives with countries that have relatively little scientific development on the topic [9, 13].

The aim of the present study is to assess the scientific literature on pneumonia that is indexed in the Web of Science (WoS). Specifically, we will analyze: (1) the evolution of scientific production; (2) its distribution by countries and regions; (3) the impact of the research papers; and (4) the degree of international collaboration. Finally, we will present details on the subject area focus of different publications according to the Medical Subject Headings (MeSH).


Identifying the population of study documents

For the performance of the study, we opted to identify documents about pneumonia by means of the MeSH thesaurus in the MEDLINE database because this is a detailed instrument for controlled terminology. The thesaurus employs both a human team of specialist indexers to analyze each article and assign medical subject headings to it, plus automated processes to improve indexing; the result is a highly consistent system of classification for research topics [14,15,16]. The pneumonia descriptor was introduced in 1963 as a disease of the respiratory tract and the lung, and it was defined as “infection of the lung often accompanied by inflammation” [17]. Synonyms of this descriptor (and therefore also included in search results) are “Lung Inflammation” and “Pulmonary Inflammation”. Additional file 1: Table S1 shows the MeSH tree structure for “Pneumonia”.

The next step was to identify the documents assigned with the MEDLINE descriptor of “pneumonia” indexed in the WoS. This body of research constitutes the population of documents for the present study. Conceived by Eugene Garfield but now maintained by Clarivate Analytics, WoS is the top scientific citation search and analytical information platform worldwide, serving both as a multidisciplinary research tool supporting a variety of scientific tasks and as a dataset for large, data-intensive studies [18].

The use of the WoS databases enables the analysis of all institutional affiliations reported in the documents and the calculation of citation indicators. The WoS brings together the most visible literature at a global level. These qualities justify its choice as the database platform used in this study despite some limitations related to covering non-English biomedical journals [18].

Although initially no limitations were imposed on our search, to calculate the bibliometric indicators we considered only two types of documents, articles and reviews, as these are the primary references for researchers. The study period was limited to 2001–2015, as delays associated with assigning MeSH descriptors to documents mean that information on the most recent articles on pneumonia is not updated. The searches took place on the Clarivate Analytics WoS platform, which includes MEDLINE database, on March 20, 2018.

Analyzing bibliographic characteristics and standardizing data

For each of the retrieved documents, data on the following bibliographic characteristics were extracted: year of publication, journal of publication and WoS subject category, document type, authorship, citations, institutional affiliation(s), and MeSH descriptors.

Data were then standardized: institutional affiliations corresponding to England, Northern Ireland, Scotland and Wales were grouped together under “United Kingdom,” while affiliations in Overseas France, British Overseas Territories, and island dependencies were also assigned to their ruling countries (for example, the documents signed by authors from French Polynesia, Guadeloupe, Martinique, New Caledonia, and Reunion were assigned to France), although regional designations correspond to geographical rather than political criteria. Scientific production from Taiwan, which in WoS is considered independently from the Democratic Republic of China (China) but whose status is disputed at an international level, was analyzed separately.

Countries responsible for publications were categorized according to their World Bank classification by income level: low-income (< USD 1025), lower-middle-income (USD 1026 to USD 4035), upper-middle-income (USD 4036 to USD 12,475), and high-income (≥ USD 12,476) countries. Each of the countries identified was assigned to a macro geographical (continental) region according to the groups established by the World Bank based on geopolitical and economic criteria and reflected in the World Bank Country and Lending Groups (see Additional file 1: Tables S2 and S3) [19].

Calculating indicators

Two kinds of indicators were obtained:

Descriptive indicators for the evolution of scientific production

We analyzed the evolution of the number of documents by year of publication and according to three 5-year periods: 2001–2005, 2006–2010, and 2011–2015. Indicators also included the frequency of publication by country, geographical region, journal and MeSH descriptor; the rate of growth in scientific production from the first to the third quinquenniums, calculated as the difference between the number of publications in 2011–2015 and those from 2001 to 2005, divided by the number of publications from 2001 to 2005.

Production by country, adjusted for demographic and economic parameters as well as for human resources dedicated to research activities

We determined standardized indicators for each country’s productivity with respect to:

  • Population: number of publications per million inhabitants (population index).

  • Gross domestic product (GDP): numbers of publications per 1 billion US dollars of GPD (GPD index).

  • Gross national income (GNI) per capita: number of publications per 100 US dollars of GNI per capita (GNI per capita index).

  • Research and development (R&D) expenditure: numbers of publications per % of GDP expenditure in R&D (R&D expenditure index).

  • Researchers in R&D: numbers of publications per researcher per million inhabitants (Researchers in R&D index)

Data were obtained from World Development Indicators in the World Bank online databases [20]. We calculated a mean value for each indicator based on available data from the study period. The analysis was limited to countries participating in the top 30 articles in the field of pneumonia in order to facilitate comparison between countries’ scientific production, demographic indicators, and economic development. Results for the top 15 articles are shown in the main text, while those for the top 30 are provided in Additional file 1.

Citation indicators

We calculated the following citation indicators by journal, country, and geographic region:

  • Citation of the publications. Absolute number of citations received.

  • Citation rate (CR). Number of citations divided by number of publications.

  • Hirsch index (h-index). The H-index is a semiqualitative proxy measure to assess the impact of an author’s or country’s research output on the scientific community [21]. An h-index of 12 indicates that 12 out of 12 published papers have been cited at least 12 times.

In order to assess the differences in the distributions of the publications according to the prestige of the journals, we performed a specific analysis of a sub-sample of publications in journals occupying the top 10% in the impact factor ranking in their respective subject categories in the Journal Citation Reports (2015 edition). We analyzed participation in these “prestigious journals” according to geographical location (regions and countries), collaboration level and number of citations.

Collaboration indicators and network analysis

We calculated the percentage of documents produced in international collaboration and the evolution by quinquennium in order to estimate the scope of cooperative practices at a global level, considering the whole population of documents analyzed (research field) by country and geographic region. To specifically analyze collaboration between countries, collaboration networks were generated for each of the three quinquenniums using Pajek software. To specifically analyze collaboration between countries, collaboration networks were generated for each of the three quinquenniums using Pajek software. The collaboration network is a graphic representation (graph), wherein the nodes represent authors’ countries (as determined from their institutional affiliations) and links between the nodes represent coauthorships between countries, that is, an international collaboration in published research. The more intense the collaboration, the thicker the links between the nodes. The spatial distribution of the nodes responds to the execution of the kamada-kawai algorithm in Pajek, which places the most prominent nodes (those with a greater number of documents and collaboration links) in the center of the map, and the nodes with a smaller number of publications and degree of collaboration towards the periphery.

Analysis of the main topics addressed in research

Based on an analysis of MeSH terms, we identified the main research focus of the studies in the area, generating density maps using the VOSviewer program with a spatial description of the main MeSH terms for each type of pneumonia [22]: (A) “Pneumonia, Aspiration” (B) “Pneumonia, Bacterial,” (C) “Pneumonia, Ventilator-Associated,” (D) “Pneumonia, Viral,” and (E) “Pneumonia, Pneumocystis”). The process of generating and interpreting the maps proceeded as follows:

  • Determination of the co-occurrence of the descriptors assigned to the documents and generation of a matrix of absolute values. The joint assignment of two descriptors in a single document implies a thematic affinity, as both aspects are addressed simultaneously in the same paper. This affinity will be more intense as it is repeated a greater number of times in the collection of documents analyzed.

  • Elimination of generic descriptors. In order to facilitate the analysis, we eliminated some excessively generic descriptors (like “humans” or “animals”), along with geographical descriptors and those related to age groups. These descriptors showed very high-density relationships, complicating the analysis and the interpretation of the results, so we analyzed their frequency more specifically.

  • Visual representation of the network. To establish the main topics that exist for each type of pneumonia and to represent them visually, we used a clustering algorithm in the VOSViewer program, which helps to detect the communities (clusters) within a network, made up of groups of homogeneous items that are strongly related to each other. The different groupings, in the form of “islands” in red tones, represent the main clusters of the thematic networks, while the chromatic gradation illustrates the areas with a lower density of relations between the MeSH in yellow and green tones. The spatial distribution of the MeSH and their proximity to each other responds to the intensity of co-occurrence between them.

All data used to perform the study, including the information downloaded from the database as well as that derived from the treatment of the bibliographic entries, are available in the Dataverse Project, an open access public repository [23] (, doi:

Ethical aspects

Due to the nature of the study and dataset, it was not necessary to obtain informed consent or approval from an institutional ethics committee.


Evolution of scientific production and distribution by country and geographic region

The search yielded a total of 33,944 documents published between 2001 and 2015 and assigned with the descriptor “Pneumonia” in the MEDLINE database. Of these, 27,017 (79.59%) were indexed in the WoS Core Collection Databases; 20,918 (77.14%) of them were classified as articles and 1776 (6.57%) as reviews. Thus, the population of study documents was a dataset of 22,694 articles and reviews, which we used to calculate the indicators presented below. Letters (N = 2213; 8.19%), editorials (N = 1, 998; 7.39%), news (N = 58; 0.21%), proceedings (N = 17; 0.06%) and other document types (N = 31, 0.11%) were excluded from the analysis.

The number of publications rose from 981 in 2001 to 1977 in 2015.The evolution of scientific production by year was fitted to a linear growth model, showing an R2 value of 0.956. Overall, the study period saw a two-fold increase in scientific production (Additional file 1: Figure S1).

The country with the greatest number of documents was the USA (38.49%), followed at some distance by the UK (7.18%), Japan (6.97%), Germany (6.80%) and France (6.73%). Table 1 shows the number of documents and the evolution of scientific production in the 15 most productive countries by quinquennium (see Additional file 1: Table S4 for results on the top 30 countries).

Table 1 Top 15 countries ranked by total number of publications by quinquenniums 2001–2005, 2006–2010, and 2010–2015

Although the USA ranks first in all periods, its relative contributions have declined, from 41.13% of all documents in 2001–2005 to 36.52% in 2011–2015. On the other hand, China’s emergence is highly notable, with a 1.13% share of total scientific production in the first period (rank = 22), compared to a 8.44% share in the third (rank = 2). South Korea has also seen considerable growth, contributing just 1.30% to total research production in 2001–2005 (rank = 19) but 3.21% in 2011–2015 (rank = 12). Likewise, Taiwan and Brazil have increased their production from 1.17 and 1.35%, respectively, to 3.02 and 3.19%.

Scientific production in different countries and geographic regions, and its evolution by quinquennium, is concentrated in North America and Europe & Central Asia; together these regions are responsible for 82.87% of the papers included in the population of documents. Research in the two regions has decreased the proportion of documents from 2001 to 2005 to 2011–2015 (− 5.46 and − 4.56%). Countries from East Asia & the Pacific and from Latin America & the Caribbean contributed with 20.90 and 4.84% of the documents, respectively. Growth was pronounced in these regions, at 13.18 and 2.52%. Table 2) (see Additional file 1: Figure S2 for a visual representation of density equalizing mapping projections).

Table 2 Geographical regions and income brackets by total number of publications and quinquennium 2001–2005, 2006–2010, and 2010–2015

Number of publications by country relative to population and economic parameters

Table 3 ranks the production of the top 15 countries, adjusted for demographic and economic indicators (see Additional file 1: Table S5 for results on the top 30 countries). When normalized by population, the most productive countries were Switzerland, the Netherlands, Iceland, and Denmark. Adjusted for the GDP index, the most productive LMICs were the Gambia, Malawi, Uganda, and Guinea Bissau. If we calculate the ratio of pneumonia publications to GNI per capita index, the USA, China, India, Malawi y Brazil were the most productive. Adjusting by R&D expenditure index, the USA ranked first, followed by Spain, the UK, China, and Italy. In relation to the researchers in R&D index, the USA also leads the ranking, followed by India, Uganda, and China. (see Additional file 1: Figure S3 and Figure S4 for a visual representation of density equalizing mapping projections of the number of documents and world development indicators, by GNI per capita index, GDP index and population index plus R&D expenditure index).

Table 3 Top 15 countries and world regions ranked according to population index, GDP index, GNI per capita index, R&D expenditure index and Researchers in R&D Indexb,a

Impact of publications

The citation analysis by geographical regions reflects the balance in the absolute number of citations received by researchers in North America and Europe, with the rest of the regions trailing considerably. In contrast, North America presents a somewhat higher citation rate (CR) than Europe (35.76 versus 29.20); among the other regions, Africa showed the best performance on this indicator (CR 31.41), with the rest presenting values of 20.07 to 24.00. In consonance with these data, at a country level the HICs (which are mostly in Europe and North America) showed higher CRs than countries in the rest of the income categories. By individual country, articles with author affiliations from the USA were the most cited (N = 316,942), followed by articles from the UK (N = 62,612), France (N = 48,019), Spain (N = 43,459) and Germany (N = 43,434). Regarding the country-specific CR, Vietnam dominated (CR 50.79), followed by the Switzerland (CR 42.94), South Africa (CR 42.85), New Zealand (CR 40.49), Saudi Arabia (CR 38.62) and the UK (CR 38.44). The USA and the UK were the top-ranked countries with an h-Index of 197 (USA) and 106 (UK), followed by France (96), Spain (94) and Germany (94) (Table 4) (see Additional file 1: Table S6 for the 30 most productive countries).

Table 4 Citation indicators for pneumonia research: rankings by 15 top-producing countries, geographic region and income (2001–2015)

Analysis of international collaboration

Overall, 18.80% of the articles published in the study period were written in international collaboration, although the rates increased from 14.35% in the 2001–2005 quinquennium to 21.64% in 2011–2015. Among the top 15 most productive countries, international collaboration was much more intense in the European countries, Brazil, Canada, and Australia (34 to 62%) compared to the USA (26.33%) and the most productive countries of East Asia & Pacific (China, South Korea, and Taiwan: 16 to 28%) (Table 5). The very high levels of international collaboration are even more pronounced in some Latin American, South Asia and particularly African countries. Indeed, the analysis of collaboration by geographical regions shows that globally, sub-Saharan Africa collaborated on 46.08% of the papers produced. The results for Latin America and the Caribbean (22.66%) are heavily weighted by research from Brazil, but the rates of international collaboration were 63.01% in Colombia, 60.94% in Argentina, and 52.21% in Mexico, while in East Asia & Pacific and South Asia (and looking beyond the most productive countries like China), countries like Bangladesh showed levels of international collaboration of 73.61%; Thailand, 60.29%; and Pakistan, 58.82%.

Table 5 Rates of international collaboration (%) in the top 15 most productive countries and by world region, pneumonia research output (2001–2015)

Figure 1 shows the collaboration networks between different countries by quinquennium. The most prominent countries in all time periods, occupying central positions in the networks with multiple cooperative links, are the USA, Canada, the UK, Germany, France, and the Netherlands. The presence of South American and African countries is scarce in all periods. Only South Africa has a notable presence in the third quinquennium (Fig. 1a). A few other countries also “emerge” with a high degree of collaborative links in the second period, like Spain, Greece, Italy, Australia, China, and Japan, although the latter two countries are not fully integrated in global networks, showing collaborative ties only with the USA (Fig. 1b). Finally, other European countries, while present throughout all three periods, stand out to a greater degree in the third period. This is the case of Sweden, Switzerland, Belgium, and Austria. At the same time, China and Japan seem more implicated in the network in this third period, while India and South Korea also gain relevance (Fig. 1c).

Fig. 1
figure 1

Networks generated from international collaborations, by quinquennium: (a) 2001–2005, (b) 2006–2010, and (c) 2011–2015

The intensity of collaboration is reflected through the thickness of the links. The most prominent nodes (those with a greater number of documents and collaboration links) are in the center of the map, while the nodes with a smaller number of publications and lower degree of collaboration are located on the periphery

Journals of publication

The documents we analyzed were published in 2115 scientific journals. Twelve journals accounted for 16.63% of the pneumonia literature Table 6

Table 6 Top 15 most productive journals and their citation indicatiors, pneumonia research 2001–2015)

. shows a list of the 15 top journals with the highest number of papers published from 2001 to 2015, as well as their impact factors for the year 2015, subject category according to the Journal Citation Reports classification, and CR (Additional file 1: Table S7 for results on the top 30 journals). The journals publishing the most articles on pneumonia were PLOS ONE (N = 494), Clinical Infectious Diseases (N = 412), and Chest (N = 397), whereas the journals with the most citations were Clinical Infectious Diseases, (N = 26,351), American Journal of Respiratory and Critical Care (N = 22,647), and Chest (N = 22,212); all of these were also among the 15 most productive journals. The journals with the highest CRs were the New England Journal of Medicine (75 documents, CR 278.13), The Lancet (54 documents, CR 210.17) and JAMA (49 documents, CR = 199.71) (see Additional file 1: Table S8 for results on the top 30 journals with highest absolute and relative citations).

The comparative analysis of the scientific production and CRs of different journals is noteworthy in that some journals (such as the American Journal of Respiratory and Critical Care, Critical Care Medicine, and Intensive Care Medicine) present a very high CR in relation to their total scientific production (Additional file 1: Figure S5 for the top 15 journals producing the most research on pneumonia, plus citation rates).

With regard to the subject categories to which the journals are assigned, the most prominent are “Infectious Diseases” (17.57% of the documents), “Respiratory System” (15.77%), “Immunology” (14.08%), “Microbiology” (11.85%), and “Critical Care Medicine” (9.26%) Table 7.

Table 7 Top 15 Web of Science Categories in pneumonia research (2001–2015)

Many of the most productive journals in pneumonia also fall into these subject categories. Moreover, over the course of the three study periods, nearly all of the subject categories saw a moderate decrease in their relative contribution, as research articles became more dispersed and made headway into different disciplines producing less research on pneumonia Table 7.

Table 8 Distribution of participation by countries in the most prestigious 10% of journals

Analysis of collaboration and citation in a top 10% de las prestigious journals

The analysis of the 4100 documents published in the top 10% of prestigious journals shows a higher participation from the USA (27.66%, compared to 38.49% in the overall body of documents) and from some other European countries like the UK or Spain. In contrast, the weight of Asian countries, particularly Japan and China, is much lower (Table 8). Overall, international collaboration in these journals (N = 1065, 25.98%) was sensibly higher than in the overall body of documents (18.8%), and the greater degree of collaboration was much more pronounced for countries like Brazil, Japan, China, and even European countries like Italy and Germany (Table 8).

The high degree of collaboration was also confirmed between regions in the publications appearing in these journals (Table 9). With regard to the degree of citation, we observed notable increases in the citation rate of the USA and the European countries; these were even more significant for countries in the Middle East & North Africa, and for sub-Saharan Africa when they participated in these journals (Table 9).

Table 9 Distribution of participation by countries in the most prestigious 10% of journals

Analysis of subject areas; frequency and distribution of MeSH terms

With regard to types of pneumonia studied, the MeSH terms to appear most frequently were “Pneumonia, Bacterial” (19.99%), followed by “Pneumonia, Pneumococcal” (7.02%), and “Pneumonia, Ventilator-Associated” (6.79%). Table 10 shows the number of documents assigned to each term describing the different types of pneumonia (Additional file 1: Table S10 for the 30 top general MeSH).

Table 10 Number of documents assigned to MeSH terms describing different types of pneumonia

Table 11 ranks the top 15 countries in crude numbers of retrieved articles, stratified by types of pneumonia (Additional file 1: Table S11 for information on the 30 most productive countries). For “Pneumonia, Aspiration”, the main countries were the USA, Japan, and Germany; for “Pneumonia, Bacterial”, the USA, France, and Spain; for “Pneumonia, Pneumocystis”, the USA, France, and the UK; for “Pneumonia, Ventilator-Associated”, the USA, France, and Spain; and for “Pneumonia, Viral”, the USA, China, and Japan.

Table 11 Distribution of research articles on different pneumonia types amont 15 most productive countries

Table 12 shows the relationship between MeSH terms referring to age groups with those corresponding to different types of pneumonia. The closest associations for “Aged, 80 and over” and “Aged” were with “Pneumonia, Aspiration” (22.58 and 40.56%, respectively), while “Pneumonia, Viral” was the most frequent topic for studies in pre-adults (“Infant”, “Child”, “Child, preschool” and “Adolescent”). The one exception to this was “Infant, newborn”, where the highest proportion of articles was about “Pneumonia, Pneumocystis.” In “Adult” and “Middle aged” people, studies most frequently focused on “Pneumonia, Bacterial” and “Pneumonia, Ventilator-Associated.”

Table 12 Distribution of MeSH terms referring to age groups, by main types of pneumonia studied in those groups

Figure 2 shows the subject area maps with the main MeSH terms in the documents on (a) “Pneumonia, Aspiration”; (b) “Pneumonia, Bacterial”; (c) “Pneumonia, Ventilator-Associated”; (d) “Pneumonia, Viral”; and (e) “Pneumonia, Pneumocystis.” The principal MeSH term related to “Pneumonia, Aspiration” is “Deglutition Disorder”, but research is linked to a broad array of topics, including epidemiological aspects (“Incidence”, “Risk Factor”, “Retrospective Studies”), treatment approaches in intensive care, and surgical techniques procedures facilitating breathing, swallowing, and feeding (Fig. 2a).

Fig. 2
figure 2

Subject area maps with the main MeSH terms associated with different types of pneumonia-(a) “Pneumonia, Aspiration” (b) “Pneumonia, Bacterial, ” (c) “Pneumonia, Ventilator-Associated, ” (d) “Pneumonia, Viral, ” and (e) “Pneumonia, Pneumocystis”

Groupings in the form of “islands” in red tones represent the main clusters of the thematic networks, while the chromatic gradation in yellow and green tones illustrates the areas with a lower density of relations between the MeSH. The spatial distribution of the MeSH and their proximity to each other responds to the intensity of co-occurrence between them

The two main MeSH terms that appear most frequently with “Pneumonia, Bacterial” are “Community-acquired Infections” and “Anti-bacterial Agents”, reflecting the central focus that research has taken to identify risk factors and test different therapeutic approaches. MeSH terms related to specific bacteria and infections, such as Streptococcus, Chlamydia, Acinetobacter, and Haemophilus influenzae, are also prominent (Fig. 2b).

For its part, research on “Pneumonia, Ventilator-associated” seems more disperse, although three areas of interest can clearly be differentiated: (a) epidemiological studies, clinical protocols, and treatment in intensive care units (the term “Intensive Care Unit” is the most prominent in this area); (b) treatment outcomes (“Treatment outcome” and “Anti-Bacterial Agents”); and (c) cross infections (“Cross infection”) (Fig. 2c).

Research on “Pneumonia, Viral” also shows a disperse nature, with different areas of interest. Epidemiological aspects are covered under terms such as “Community-acquired Infections” and “Hospitalization”, while at a researcher level, interests reside in the virus “Influenza, Human” and “Orthomyxoviridae Infections” (Fig. 2d). With regard to “Pneumonia, Pneumocystis”, one prominent subject focus is on “AIDS-Related Opportunistic Infections” and another is on “Pneumocystis jirovecii” (Fig. 2e).


Our analysis shows that the number of publications on pneumonia increased notably over the study period, with annual research outputs doubling from 2001 to 2015. Different factors may have contributed to this. The first of these is the growing research relevance of pneumonia as a clinical entity, as this disease is one of the community-acquired infections with the highest incidence and is an important cause of hospital admissions. It is also associated with a high global burden of morbidity and mortality in both children and adults [1,2,3, 24]. The second potential factor relates to advances in basic immunological and microbiological research along with deepening knowledge on the pathogenesis of the disease with regard to aspects like microbiological resistance and preventive interventions (e.g. vaccines) [25]. Thirdly, increased funding has been directed toward research and particularly “proactive investments for emerging infectious threats” [8, 9], and finally, the increase in scientific production could be related to scientific development and international dissemination of scientific research in the WoS databases. This is particularly the case of China and other emerging economies like Brazil, where the rates of growth were highest relative to their respective regions [26,27,28].

We observed a substantial increase in research worldwide, but particularly in some geographical regions and countries of South Asia, East Asia & the Pacific, Latin America & and the Caribbean, and sub-Saharan Africa. To a great extent, this increase is simply a reflection of the limited contribution to global research that these countries made in the first period analyzed (2001–2005). The bulk of scientific production continues to come from countries with more economic and scientific development in Europe and North America (together, these countries participated in 77% of all publications).

Despite the striking increase in scientific production across LMICs, the relative contribution to pneumonia research remains very modest, and the fact that some countries rank highest in demographic and economic indicators may not be a positive feature, but rather a reflection of the scant development in their scientific systems. Furthermore, the increase in international collaboration could have played a role in these indicators, multiplying the assignment of articles to different countries and possibly inflating some values, masking the real contribution of countries with less scientific development in research activities [29].

The USA is undoubtedly the main reference for pneumonia researchers in quantitative terms, as it produces by far the largest volume of publications—four times that of the next most productive country in the last period. Other European countries with important scientific systems (e.g. the UK, Germany, France, and Spain), along with other countries like Japan, Canada, China, India, and Brazil, also stand out in relation to some of the indicators of scientific production and economic development (GNI per capita index, and R&D expenditure Index). The other significant aspect in the analysis of how scientific production evolved over the study period is the emergence of China, which in the last period of study (2011–2015) trailed only the USA in research output. This growth has come about in large part from the investments and scientific policies to foster openness that have been implemented over the past several decades to promote internationalization [30, 31].

The level of international scientific collaboration that we have observed in the field of pneumonia (19%) is below that seen in other areas of knowledge [11, 29, 30, 32,33,34,35]. Thus, even though the trend is toward increased international cooperation, rising from 14 to 22% over the study period, implementing new strategies that favor collaboration is still necessary [11].

Initiatives promoting research could include those launched by international organizations, such as the World Health Organization (WHO) and the Bill & Melinda Gates Foundation, which have both invested considerable resources to investigate the etiology of childhood pneumonia in low-income countries [36,37,38]. However, these initiatives carry risks too, as major actors in LMIC research, including the Bill & Melinda Gates Foundation, have been shown to be biased toward research done by researchers from HIC (doing research in LMIC) [39].

The European and Developing Countries Clinical Trials Partnership and the Global Fund [40] are also collaborating in different projects related to HIV, tuberculosis, and malaria, and these organizations are largely responsible for the important degree of collaboration between European and sub-Saharan African countries [41]. Research for operational health services is necessary to improve the distribution and accessibility of pneumonia treatments, including antibiotics in primary healthcare centers and oxygen in hospitals. Likewise, new vaccines still need to be developed for strains of pneumococcus that current multivalent conjugate vaccines do not protect against [8].

In addition to programs focused on financing and implementing collaborative North-South and South-South projects, other efforts could be directed toward reducing obstacles associated with publication processes that limit the dissemination of LMICs through the main international scientific journals. The literature has described obstacles related to linguistic skills and methodological deficiencies, which highlights the need to improve these areas in particular [42, 43]. Other authors have pointed to the costs associated with publishing in open access journals, so it is worth assessing whether the programs to support open access publishing implemented at an institutional level and by publishers such as PLOS, Biomed Central, or The Lancet Journals, are sufficient [44,45,46].

With regard to the impact of research, although Europe and North America are balanced in terms of the absolute number of citations, North America holds an advantage in terms of the citation rate. Research from sub-Saharan Africa also has a very high citation rate, which almost reaches that achieved in Europe. The fact that these African countries present a high degree of collaboration with researchers in the USA and Europe, who represent the “mainstream” international research interests, could help explain the high citation rates seen in this region. On the other hand, Latin America & Caribbean, South Asia, and East Asia & Pacific are all regions with generally lower citation rates, although this difference is not so pronounced in the case of papers produced in collaboration, as reported elsewhere [47].

By country, the hegemony of the USA and several European countries in terms of the number of citations received was evident, as was the lower ranking of some Asian countries, such as Japan and China, in relation to their scientific production. The positioning of China as a reference for scientific production and participation in international research networks does not correspond to its ranking with regard to citation indicators, despite their improved standing over the past several years [30]. On these indicators, China still lags behind the USA as well as the leading European countries, Canada, Australia and even nearby countries such as Japan. For now at least, the countries that have traditionally occupied the “mainstream” of scientific research still maintain their hegemony [48].

As with the relative indicators of scientific production adjusted for economic and demographic parameters, some countries surpass the major scientific systems with regard to the citation rate, which links the degree of citation with the volume of scientific production [33]. These countries may have participated in certain highly relevant contributions, or they may be small countries with highly developed scientific systems, such as Vietnam, Switzerland, South Africa, New Zealand, and Saudi Arabia. These countries also stand out for their high levels of international collaboration, which is a factor associated with more citations.

The high mean citations received by publications produced in sub-Saharan Africa, and the participation of different emerging countries like Vietnam and South Africa in some of the highest cited papers we identified, underlines the capacity of these countries to contribute to high-impact and excellent-quality scientific studies. This result is consistent with previous studies that have also demonstrated these countries’ capacity to participate in emerging research topics [49]. These specialists therefore represent an excellent asset, strengthening the human capital from high-income countries and enabling the advancement of research [50, 51].

In general, the most prestigious journals show a greater concentration of research from the USA and Europe, with greater collaboration and impact when countries from other geographical regions also participate [52].

Bacterial pneumonia is the main branch for the multidisciplinary and multipathological MeSH of “Pneumonia”, with the main areas of interest (“Community-acquired Infections”, “Anti-bacterial Agents” and “Treatment Outcome”) reflecting the focus of research on identifying risk factors and assessing different treatments and their outcomes. In publications pertaining to the MeSH “Pneumonia, Ventilator-Associated,” the main axes of the subject content according to the MeSH terms were the group of epidemiological studies and clinical and treatment protocols in intensive care. “Pneumonia Pneumocystis,” is closely related to infection due to HIV and immunodepression. The main areas of research interest for “Pneumonia, Viral,” were the epidemiological aspects related to the setting for the infection (“Community-acquired Infections” and “Hospitalization”) along with the viruses responsible (“Influenza, Human” and “Orthomyxoviridae Infections”). Finally, for the MESH “Pneumonia, Aspiration” the main research focus is “Deglutition Disorder”.

The main limitation of this present study is its analysis of only the documents included in the WoS databases and MEDLINE (80% of the documents). Thus, a number of papers were excluded from the study, particularly those written in languages other than English, as well as the proceedings included in WoS, as our searches were based on the journals included in MEDLINE. On the other hand, our approach also allowed us to precisely characterize collaboration in the area, as only recently has MEDLINE begun to include all the institutional affiliations of the authors. We were also able to analyze the citations of the publications, with a focus on the journals with the highest impact and dissemination at an international level [28].

In conclusion, pneumonia research increased steadily over the 15-year study period, with Europe and North America leading scientific production. About a fifth of all papers reflected international collaborations, and these were most evident in papers from sub-Saharan Africa and South Asia.

Availability of data and materials

All data used to perform the study, including the information downloaded from the database as well as that derived from the treatment of the bibliographic entries, are available in the Dataverse Project, an open access public repository [23] (, doi:


  1. Welte T, Torres A, Nathwani D. Clinical and economic burden of community-acquired pneumonia among adults in Europe. Thorax. 2012;67:71–9.

    Article  CAS  PubMed  Google Scholar 

  2. File TM, Marrie TJ. Burden of community-acquired pneumonia in north American adults. Postgrad Med. 2010;122:130–41.

    Article  PubMed  Google Scholar 

  3. GBD 2015 Mortality and Causes of Death Collaborators H, Naghavi M, Allen C, Barber RM, Bhutta ZA, Carter A, et al. Global, regional, and national life expectancy, all-cause mortality, and cause-specific mortality for 249 causes of death, 1980–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet. 2016;388:1459–544.

    Article  Google Scholar 

  4. Cillóniz C, Polverino E, Ewig S, Aliberti S, Gabarrús A, Menéndez R, et al. Impact of age and comorbidity on cause and outcome in community-acquired pneumonia. Chest. 2013;144:999–1007.

    Article  PubMed  Google Scholar 

  5. Ochoa-Gondar O, Vila-Córcoles A, de Diego C, Arija V, Maxenchs M, Grive M, et al. The burden of community-acquired pneumonia in the elderly: the Spanish EVAN-65 study. BMC Public Health. 2008;8:222.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Torres A, Cillóniz C, Blasi F, Chalmers JD, Gaillat J, Dartois N, et al. Burden of pneumococcal community-acquired pneumonia in adults across Europe: a literature review. Respir Med. 2018;137:6–13.

    Article  PubMed  Google Scholar 

  7. World Health Organization. All cause mortality estimates for 2000–2016. WHO. 2018. Accessed 16 June 2018.

  8. Head MG, Fitchett JR, Newell M-L, Scott JAG, Harris JN, Clarke SC, et al. Mapping pneumonia research: a systematic analysis of UK investments and published outputs 1997-2013. EBioMedicine. 2015;2:1193–9.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Head MG, Fitchett JR, Cooke MK, Wurie FB, Hayward AC, Lipman MC, et al. Investments in respiratory infectious disease research 1997–2010: a systematic analysis of UK funding. BMJ Open. 2014;4:e004600.

    Article  PubMed  PubMed Central  Google Scholar 

  10. González-Alcaide G, Salinas A, Ramos JM. Scientometrics analysis of research activity and collaboration patterns in Chagas cardiomyopathy. PLoS Negl Trop Dis. 2018;12:e0006602.

    Article  PubMed  PubMed Central  Google Scholar 

  11. González-Alcaide G, Park J, Huamaní C, Belinchón I, Ramos JM. Evolution of Cooperation Patterns in Psoriasis Research: Co-Authorship Network Analysis of Papers in Medline (1942–2013). PLoS One. 2015;10:e0144837.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Ramos JM, González-Alcaide G, Bolaños-Pizarro M. Bibliometric analysis of leishmaniasis research in Medline (1945-2010). Parasit Vectors. 2013;6:55.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Head MG, Fitchett JR, Newell M-L, Scott JAG, Clarke SC, Atun R. Investment in pneumonia and pneumococcal research. Lancet Infect Dis. 2014;14:1037–8.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Coletti MH, Bleich HL. Medical subject headings used to search the biomedical literature. J Am Med Inform Assoc. 2001;8:317–23.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Mao Y, Lu Z. MeSH now: automatic MeSH indexing at PubMed scale via learning to rank. J Biomed Semantics. 2017;8:15.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Mork J, Aronson A, Demner-Fushman D. 12 years on – is the NLM medical text indexer still useful and relevant? J Biomed Semantics. 2017;8:8.

    Article  PubMed  PubMed Central  Google Scholar 

  17. National Center for Biotechnology Information. U.S. National Library of Medicine. MeSH datebase: Pneumonia. 1963. Accessed 16 July 2018.

  18. Li K, Rollins J, Yan E. Web of science use in published research and review papers 1997-2017: a selective, dynamic, cross-domain, content-based analysis. Scientometrics. 2018;115:1–20.

    Article  PubMed  Google Scholar 

  19. The World Bank. World Bank Country and Lending Groups. World Bank Data Help Desk. Accessed 16 June 2018.

  20. The World Bank. World Bank Open Data. The World Bank. Accessed 16 June 2018.

  21. Hirsch JE. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci. 2005;102:16569–72.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. 2010;84:523–38.

    Article  PubMed  Google Scholar 

  23. Ramos J, Pinargote H, Belinchón I, González-Alcaide G. Replication data for: mapping global pneumonia research: a systematic analysis of scientifical production (2001–2015); 2018.

    Google Scholar 

  24. Liu L, Johnson HL, Cousens S, Perin J, Scott S, Lawn JE, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet (London, England). 2012;379:2151–61.

    Article  Google Scholar 

  25. Park H, Li Z, Yang XO, Chang SH, Nurieva R, Wang Y-H, et al. A distinct lineage of CD4 T cells regulates tissue inflammation by producing interleukin 17. Nat Immunol. 2005;6:1133–41.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Woo PCY, Lau SKP, Chu C, Chan K, Tsoi H, Huang Y, et al. Characterization and complete genome sequence of a novel coronavirus, coronavirus HKU1, from patients with pneumonia. J Virol. 2005;79:884–95.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, web of science, and Google scholar: strengths and weaknesses. FASEB J. 2008;22:338–42.

    Article  CAS  PubMed  Google Scholar 

  28. Powell KR, Peterson SR. Coverage and quality: a comparison of web of science and Scopus databases for reporting faculty nursing publication metrics. Nurs Outlook. 2017;65:572–8.

    Article  PubMed  Google Scholar 

  29. Persson O, Glänzel W, Danell R. Inflationary bibliometric values: the role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics. 2004;60:421–32.

    Article  CAS  Google Scholar 

  30. Bornmann L, Wagner C, Leydesdorff L. BRICS countries and scientific excellence: a bibliometric analysis of most frequently cited papers. J Assoc Inf Sci Technol. 2015;66:1507–13.

    Article  CAS  Google Scholar 

  31. Zhou P, Glänzel W. In-depth analysis on China’s international cooperation in science; 2010.

    Book  Google Scholar 

  32. González-Alcaide G, Park J, Huamaní C, Ramos JM. Dominance and leadership in research activities: collaboration between countries of differing human development is reflected through authorship order and designation as corresponding authors in scientific publications. PLoS One. 2017;12:e0182513.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Brüggmann D, Köster C, Klingelhöfer D, Bauer J, Ohlendorf D, Bundschuh M, et al. Respiratory syncytial virus: a systematic scientometric analysis of the global publication output and the gender distribution of publishing authors. BMJ Open. 2017;7:e013615.

    Article  PubMed  PubMed Central  Google Scholar 

  34. González-Alcaide G, Ramos JM, Huamaní C, de Mendoza C, Soriano V. Human T-lymphotropic virus 1 (HTLV-1) and human T-lymphotropic virus 2 (HTLV-2): geographical research trends and collaboration networks (1989-2012). Rev Inst Med Trop Sao Paulo. 2016;58:11.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Groneberg DA, Weber E, Gerber A, Fischer A, Klingelhoefer D, Brueggmann D. Density equalizing mapping of the global tuberculosis research architecture. Tuberculosis. 2015;95:515–22.

    Article  PubMed  Google Scholar 

  36. Levine OS, O’Brien KL, Deloria-Knoll M, Murdoch DR, Feikin DR, DeLuca AN, et al. The Pneumonia Etiology Research for Child Health Project: a 21st century childhood pneumonia etiology study. Clin Infect Dis. 2012;54 Suppl 2 suppl 2:S93–101.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Bill and Melinda Gates Foundation. 2015 Gates annual letter. Our Big Bet For The Future Gatenotes 2015. Accessed 16 June 2018.

  38. World Health Organization. Priority Medicines for Europe and the World 2013 Update. Background Paper 6 - Priority Diseases and Reasons for Inclusion. BP 6.15 - Depression. WHO. 2013. Accessed 16 July 2018.

  39. McCoy D, Kembhavi G, Patel J, Luintel A. The bill &amp; Melinda Gates Foundation’s grant-making programme for global health. Lancet. 2009;373:1645–53.

    Article  PubMed  Google Scholar 

  40. Matee MI, Manyando C, Ndumbe PM, Corrah T, Jaoko WG, Kitua AY, et al. European and Developing Countries Clinical Trials Partnership (EDCTP): the path towards a true partnership. BMC Public Health. 2009;9:249.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Breugelmans JG, Makanga MM, Cardoso ALV, Mathewson SB, Sheridan-Jones BR, Gurney KA, et al. Bibliometric assessment of European and sub-Saharan African research output on poverty-related and neglected infectious diseases from 2003 to 2011. PLoS Negl Trop Dis. 2015;9:e0003997.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Singh D. Publication bias - a reason for the decreased research output in developing countries. Afr J Psychiatry. 2006;9:153–5.

    Article  Google Scholar 

  43. Uthman OA, Uthman MB. Geography of Africa biomedical publications: an analysis of 1996–2005 PubMed papers. Int J Health Geogr. 2007;6:46.

    Article  PubMed  PubMed Central  Google Scholar 

  44. PLOS. Secondary publication fee assistance. Accessed 5 Aug 2018.

  45. Bio Med Central. BMC: APC waivers and discounts. Accessed 5 Aug 2018.

  46. BMJOpen. BMJ open: Secondary instructions for authors. Accessed 16 July 2018.

  47. Meneghini R, Packer AL, Nassi-Calò L. Articles by latin american authors in prestigious journals have fewer citations. PLoS One. 2008;3:–e3804.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Ding Z-Q, Ge J-P, Wu X-M, Zheng X-N. Bibliometrics evaluation of research performance in pharmacology/pharmacy: China relative to ten representative countries. Scientometrics. 2013;96:829–44.

    Article  Google Scholar 

  49. Glänzel W, Zhang L. Scientometric research assessment in the developing world: a tribute to Michael J. Moravcsik from the perspective of the twenty-first century. Scientometrics. 2018;115:1517–32.

    Article  Google Scholar 

  50. Zdravkovic M, Chiwona-Karltun L, Zink E. Experiences and perceptions of south--south and north--south scientific collaboration of mathematicians, physicists and chemists from five southern African universities. Scientometrics. 2016;108:717–43.

    Article  Google Scholar 

  51. Confraria H, Godinho MM. The impact of African science: a bibliometric analysis. Scientometrics. 2015;102:1241–68.

    Article  Google Scholar 

  52. Akre O, Barone-Adesi F, Pettersson A, Pearce N, Merletti F, Richiardi L. Differences in citation rates by country of origin for papers published in top-ranked medical journals: do they reflect inequalities in access to publication? J Epidemiol Community Health. 2011;65:119–23.

    Article  PubMed  Google Scholar 

Download references


We gratefully acknowledge the assistance of Meggan Harris in translating our manuscript from Spanish.


No funding was received for this work.

Author information

Authors and Affiliations



JMRR: study conception, study, design, data analysis, manuscript writing and final manuscript approval; HPC: data collection, data analysis, manuscript writing and final manuscript approval; IBR: study conception, manuscript writing and final manuscript approval; GGA: study conception, study design, data collection, data analysis, manuscript writing and final manuscript approval

Corresponding author

Correspondence to José M. Ramos-Rincón.

Ethics declarations

Ethics approval and consent to participate

Due to the nature of the study and dataset, it was not necessary to obtain informed consent or approval from an institutional ethics committee.

Consent for publication

The authors give consent to publish the manuscript.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Table S1. Descriptors included under the MeSH “Pneumonia” in PubMed. Table S2. Countries by regions according to World Bank Country and Lending Groups. Table S3. Countries by incomes according to World Bank Country and Lending Groups. Table S4. Top 30 countries ranked by total number of publications by quinquennium 2001–2005, 2006–2010 and 2011–2015. Table S5. Top 30 countries and world regions ranked according to according to population index, GDP index, GNI per capita index, R&D expenditure index and Researchers in R&D Index. Table S6. Top 30 countries ranked according to citations, citation rate and h-Index in the period 2001–2015. Table S7. Top 30 journals with the highest number of pneumonia articles published in 2001–2015, citations, citation rate (CR), impact factors for the year 2015, journal category with ranking from the Journal Citation Report and language of publication. Table S8. Top 30 journals with citations and citations rate (CR). Table S9. Top 30 citations rate (CR) journal *. Table S10. The 30 top general Medical Subject Headings (MeSH). Table S11. Top 30 countries in crude numbers of retrieved articles in “Pneumonia, Aspiration”, “Pneumonia, Bacterial”, “Pneumonia Pneumocystis”, “Pneumonia, Ventilator-Associated”, and “Pneumonia, Viral” MeSH. Figure S1. Evolution of scientific production on pneumonia (2001–2015). Figure S2. Density equalizing mapping projections. Number of documents per quinquennium for scientific production on pneumonia, (A) 2001–2005; (B) 2006–2010, and (C) 2011–2015. Figure S3. Density equalising mapping projections: number of documents and world development indicators, (A) GNI per capita index; (B) GDP index. Figure S4. Density equalising mapping projections: number of documents and world development indicators (A) population index; (B) R&D expenditure index. Figure S5. Top 15 journals producing the most research on pneumonia, plus citation rates. (DOCX 7194 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramos-Rincón, J.M., Pinargote-Celorio, H., Belinchón-Romero, I. et al. A snapshot of pneumonia research activity and collaboration patterns (2001–2015): a global bibliometric analysis. BMC Med Res Methodol 19, 184 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: