Bibliometric Analysis to Explore Trends of the 100 Most Cited Articles in Population Pharmacokinetic and/or Pharmacodynamic Modelling

Understanding the dose-concentration-effect relationship is a fundamental concept in clinical pharmacology. Mathematical and statistical models are often used to help describing the above mentioned relationship.^[1] More specifically, such models are: population pharmacokinetic (PK) models [to describe the relationship between drug dose and concentration], pharmacodynamic (PD) models [to describe the time course of the biomarkers/safety/efficacy], and pharmacokinetic-pharmacodynamic (PKPD) models [incorporates both PK and PD to describe the relationship between drug dose and/or concentration and effect].^[2] PKPD modelling and simulation (M&S) plays an important role in drug development and is promoted by the regulatory agencies, including the United States Food and Drug Administration (FDA) and European Medicines Agency (EMA) to improve efficiency and productivity in pharmaceutical drug development.^[3–5]

Several methods for determining drug population pharmacokinetics have been proposed, such as the naïve average data approach, naïve pooled data analysis, a two-stage approach, and a Nonlinear Mixed-Effects (NLME) modelling approach.^[6] Human patients differ from one another, therefore their drug disposition and pharmacodynamic effects are heterogenous. To be able to describe the PK (and PD) parameters without bias, an approach that does not over- or underestimate such between patient variability is needed. Comparing the above-mentioned modelling approaches, the mixed-effects modelling approach provides estimates of such variability that are more accurate and less biased.^[6] The NLME approach is implemented in many general and specialised statistical tools^[7,8] and it is the most widely used population-based modelling,^[9,10] possibly due to its advantage of being able to handle sparse and unbalanced data (i.e. data where the number of observations and/or sampling times differs between individuals) better compared to other approaches. Furthermore, an NLME model can quantitatively describe the effects of covariates (e.g., sex, age, body weight) on the PK data.^[11,12] Thus, this study only focused on the analyses that used the NLME approach.

Bibliometric analysis is a scientific technique that identifies central concepts, emerging trends, and areas of study interest in certain subjects or research domains by fusing statistical techniques with information visualisation technologies.^[13] Citation counts have long been used to assess the quality of research articles and the contributions and scientific performance of scientists, research groups, departments, and universities. These data have been frequently used to make, for example, key policy, performance, career advancement, award selection, and funding decisions.^[14] Furthermore, bibliometric analysis allows researchers to identify key study topics and explore the trends in a field, and thus providing ideas and directions for future research.^[15]

Hitherto, there has been no bibliometric analysis or other analysis, focusing also on the research trends of most cited articles in the population PK, PD, or PKPD modelling. Hence, we designed a bibliometric study to analyse the top 100 most cited articles since inception (1964-2021) and in the recent years (2015-2021) to explore the trends of most cited papers using data from the Scopus database.

Information on the 100 most cited articles on population NLME PK (/PD) modelling since inception and in the recent years was obtained from the Scopus online database. The search for articles on population modelling, specifically PK or PD or PKPD modelling studies, was restricted to the article title, abstract, keywords, and studies published by the end of 2021. The Scopus database was accessed in February 2022, to perform searches for overall top 100 most cited articles, and again in August 2022, to search for publications published between 2015-2021. The following query string was used: TITLE-ABS-KEY (“population pharmacokinetic* model*” OR “pharmacokinetic* model*” OR “population pharmacokinetic*” OR “population pharmacodynamic*” OR “pharmacodynamic* model*” “pharmacokinetic* pharmacodynamic* model*” OR “nonlinear mixed effect* model*”) AND PUBYEAR < 2022.

Key data from the Scopus database were merged with data from manual search approaches to ensure data accuracy and comprehensiveness. Information collected from the Scopus database included the article’s title, abstract, keywords, publication year, publication types, the number of citations, name of authors, authors’ countries, organisations, journals, and funding. Furthermore, information from journal quartiles and Anatomical Therapeutic Chemical (ATC) Classification were retrieved manually to supplement the Scopus information. In this regard, we referenced the SCImago Journal Rank (SJR) 2021 edition^[16] to determine journal quartiles, while the therapeutic class was identified based on the Anatomical Therapeutic Chemical (ATC) Classification system at the first (main anatomical group) and second level (drug classes).

Each publication was reviewed to ensure that the study refers to the population PK or PD or PKPD modelling rather than other fields like software development. Full texts of the articles were retrieved where necessary.

The full inclusion criteria are outlined below:

1. Original articles related to the development, evaluation or validation, and application of population pharmacokinetic and/or pharmacodynamic models.
2. Population data were analysed using a Nonlinear Mixed Effect (NLME) modelling approach.

The exclusion criteria included reviews, methodology articles, and preclinical in vitro and in vivo studies. Figure 1 describes the selection process for the years 1964-2021 and Supplementary Figure 1 describes the selection process for the years 2015-2021 based on the PRISMA 2009 flow diagram.^[17] No ethics approval was required since this study was a bibliometric analysis with no human/animal subjects involved.

Figure 1:
PRISMA flowchart illustrating the selection process of the 100 most cited articles about PK(/PD) modelling between 1964 and 2021.
* Articles with <150 citations were excluded to ensure only top 100 most cited articles remained
PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Harzing’s Publish or Perish software v8 was used to import data from the Scopus database to determine each article’s citations per year. To map and visualise the co-authorship relationship between countries of the authors we produced network maps via VOSviewer 1.6.15 software (www.vosviewer.com).

The number of citations ranged from 137 to 914 and 37 to 156 for the 100 most cited articles since inception and in the recent years, respectively. Most of the 100 most cited articles since inception were published in 2008 (n=10) involving anti-infectives, central nervous system agents, and chemotherapeutics and immunomodulating agents, while most articles in the recent years were published in 2015 (n=45) with a similar scope of studies. The Supplementary Table 1 lists the 100 most cited articles on population PK, PD, or PKPD modelling since inception to 2021 based on the number of citations in descending order. Meanwhile, the most cited articles published 2015-2021 are listed in Supplementary Table 2.

The article “Influence of age and gender on the pharmacokinetics and pharmacodynamics of remifentanil I. Model development”^[18] has the highest number of total citations since inception until 2021 (n=914). Whereas “Population pharmacokinetics of colistin methane sulfonate and formed colistin in critically ill patients from a multicenter study provide dosing suggestions for various categories of patients”^[19] has the highest citation rate per year with an average of 48.82 citations per year (Supplementary Table 1). Eighty-eight percent of the 100 most cited articles since inception were published in Quartile 1 journals, which increased to 91% in recent years.

The most studied ATC class of drugs in the top 100 most cited articles were chemotherapeutics and immunomodulating agents (33%), with anti-infectives (29%) and central nervous system ATC class (22%) represented similarly in the second and third place (Figure 2). In the recent 100 most cited articles the trends remained similar, with the number of studies analysing the chemotherapeutic and immunomodulating agents, and anti-infective ATC drug class increasing (42% and 39%, respectively) whereas the number of central nervous system ATC class reduced (9%). Detailed information on each ATC group and the respective drug classes is summarised in Figure 2.

Figure 2:
Trends in Anatomical Therapeutic Classification (ATC) groups/drug classes in top 100 most cited articles since inception, and in the recent years (2015-2021).
Some studies did not have an ATC number as they are under chemical/plants classification, hence they were not included in the analysis. A total of 93 studies with an ATC number were included in Figure (a), and 95 studies for Figure (b).

Antineoplastic agents (23%), antibacterials (13%) and anaesthetics (11%) were the top 3 drug classes in the 100 most cited articles since inception. However, the percentage of influential papers publishing antibacterial modelling results increased recently to 20%, with antineoplastic agents staying relatively the same (24%) and increasing to third place is immunosuppressant drugs (18%) (Figure 2). Detailed information for these drug classes is listed in Supplementary Tables 3-5. The most significant changes within the impactful articles reporting results on antineoplastic, antibacterial, and immunosuppressant drug classes are summarised below, other than that there have not been any major research trend changes. The proportion of studies in paediatric population increased in all the three drug classes. Additionally, most antibacterial studies shifted from polymyxins (42%) to glycopeptide antibiotics (37%) in the recent years (Supplementary Table 4) and for immunosuppressant drugs the trend changed from tumour necrosis factor-alpha inhibitors (44%) to selective immunosuppressants (53%) (Supplementary Table 5).

Most of the studies described in the top 100 most cited articles were population PK studies (65%) followed by population PKPD modelling studies (30%) (Table 1). Recently, the number of population PK modelling studies decreased (55%) and the number of populations PKPD modelling studies increased (43%). PD studies represented only ≤5% of the studies throughout the years (Table 1).

The classification of PKPD modelling study was determined if the article used at least one population PK and/or PD model, e.g. a study of population PK, combined with a PKPD metric for the PD part was counted as a PKPD modelling study.

Majority of the impactful articles reported analyses of the adult population compared to the paediatric populations (Table 2), with only a small increase in the percentage (1%) of studies performed in the paediatric population in recent years. In the 100 most cited articles since inception most paediatric studies included children over 2 years (7 out of 12 paediatric studies), however, recently, most studies (6 out of 13 paediatric studies) included a wide distribution of paediatric ages (i.e. from newborn until adolescent) (Table 2).

Eighty percent of the top 100 most cited articles in all publications used NONMEM® software to analyse the PK(/PD) data. Interestingly, this number recently increased (83%) confirming that NONMEM® is still the main software for population analyses (Table 3). Within the most cited research, only a few researchers used different types of software besides NONMEM®, such as, S-ADAPT, NLME, Phoenix NLME, Monolix, SAS software, NPEM2 and NLMIXED (Table 3).

The number of authors ranged from 1 to 20 (median 8 authors) and 3 to 32 (median 9 authors) for the 100 most cited publications and the top 100 most cited publications in recent years, respectively.

Table 4 shows the top six institutions that were affiliated with authors who published the 100 most cited population PK, PD or PKPD modelling analyses. Out of these, three institutions were from the European Union and two were from the US, however, recently, 5 out of the top 6 institutions (except University of Queensland) were from Europe. The institution that contributed the most to the 100 most cited articles since inception was Uppsala University (8%), but recently, University of Liverpool and Leiden University Medical Center both contributed as much as Uppsala University (6%) (Table 4).

The 100 most cited articles were published in 39 academic journals and 38 academic journals for overall years and recent years, respectively. The majority of these impactful articles since inception were published in Anesthesiology, however, recently, the impactful research has been mostly published in Antimicrobial Agents and Chemotherapy (Table 5). Top 5 journals that published more than 40% of the 100 most cited articles since inception and in recent years, remained mostly unchanged over time. Nevertheless, in the recent years, Clinical Pharmacokinetics entered the top 5, replacing Anesthesiology. Detailed information on the top journals and their SCImago Journal Ranking (SJR) value is given in Table 5.

Co-authorship network analyses showed that the circle representing United States was the largest indicating the most publications (Figures 3 and 4). United States also had the highest total link strength of 54 (Table 6) which even increased recently (to 65), making USA the most prominent collaborating country. The Netherlands and United Kingdom remained in the top three most prominent countries. The collaboration between countries has, however, changed slightly over time. In the overall 100 most cited articles researchers collaborated most with and between developed countries, such as, United States and Germany or Sweden (Figure 3), but recently, an increasing number of researchers from developing countries (such as South Africa and China) collaborated with those from a developed country (Figure 4).

Figure 3:
Network visualisation map of the co-authorship between countries for the top 100 most cited publications between 1964 and 2021.
Each circle represents a country, and a line connecting the circles indicates countries that appeared as the authors’ countries together in any of the top 100 most cited articles. A larger circle represents a higher number of articles. The link strength attribute indicates the numbers of collaboration between different countries. A higher total link strength indicates a higher collaboration between these countries.

Figure 4:
Network visualisation map of the co-authorship between countries for the top 100 most cited publications in the recent years (2015-2021).
Each circle represents a country, and a line connecting the circles indicates countries that appeared as the authors’ countries together in any of the top 100 most cited articles. A larger circle represents a higher number of articles. The link strength attribute indicates the numbers of collaboration between different countries. A higher total link strength indicates a higher collaboration between these countries.

Table 7 summarises the funding status for the 100 most cited articles. Unfortunately, not all publications reported the founding source: in the top 100 most cited publications 20 did not report it, however, this improved with time (to n=7). More than 70% of the studies received funding locally since inception, but in the top 100 most cited articles published recently, there was a slight increase (2%) of international funding (Table 7). As an example, one study from a developing country (South Africa) received funds from the World Health Organization, and the United States Agency for International Development.^[21]

To the best of our knowledge, this is the first study to summarise the top 100 most cited articles in the field of population PK, PD or PKPD modelling using the bibliometric approach. Moreover, we also showed how influential research landscape changed over time in the aforementioned field.

It has been suggested that articles that exceed the threshold of 100 citations can be considered “classic articles”, a sort of “milestones” in the development of a specific area.^[22,23] Since the number of citations for the 100 most cited articles for overall publications exceeds 100 citations, all the articles listed for overall publications can be considered “classic”. Knowledge and understanding of these classic papers can help guide education and provide additional insights into a better understanding of a specific field.^[24] This study also found that during the recent years, there has been an increase in the number of publications published in Q1 journals, indicating that over time high impact journals became even more impactful.

This study found that the majority of the impactful articles used population PK modelling to analyse data. This is not surprising as PK data are in many cases easier to collect/measure and evaluate than PD data. Additionally, population PK is an established field with a specific regulatory guidance and many methodological articles on model development and diagnostics.^[25,26] PD results can, however, depend strongly on the disease, and the techniques for their analysis can thus be more difficult to unify. Still, recently, researchers started giving more emphasis to PD, especially to how PK affects PD; which is reflected also in a greater proportion of PKPD studies within the impactful articles. This may also be a consequence of regulatory recommendations, e.g. in 2016, the European Medicines Agency (EMA) published the “Guideline on the use of pharmacokinetics and pharmacodynamics in the development of antibacterial medicinal products^[3] that focuses on the use of PKPD analyses to identify potentially efficacious dosing regimens.

Recently, in the top 100 most cited articles, there have been more studies looking at antibacterial and immunosuppressant classes of drugs (Figure 2). In the anti-infectives ATC classification, antimicrobials had the largest increase, possibly another consequence of the aforementioned EMA guidance^[3] on the emerging research in this field. The increase was most prominent in special patient populations, where research was/is lacking the most, such as, the critically ill patients,^[27,28] extremely obese patients^[29] or neonates.^[30–32] In the chemotherapeutic and immunomodulating ATC class, most articles analysed PK(/PD) data of antineoplastic agents, which have remained the most cited over time. This may be due to cancer treatment being a continued important area of interest for the pharmaceutical industry, and one of the fastest growing areas in personalised medicines.^[33] Over time, monoclonal antibodies (mAbs) have remained the most researched antineoplastic drugs in the top 100 most cited articles. This is in line with mAbs becoming a significant class of therapeutic drugs available;^[34] to date, there are more than 100 mAbs approved by the US Food and Drug Administration.^[35]

NONMEM® software was the most widely used software, which remained unchanged over the years. This is expected, given that NONMEM® was the first software package developed by Lewis Sheiner and Stuart Beal to handle complex PK calculations in the nonlinear mixed effect modelling approach.^[36,37] Furthermore, it is widely regarded as the gold standard in population pharmacokinetic-pharmacodynamic modelling.^[38,39] Recently, the percentage of the influential articles that used NONMEM® increased further, despite other emerging software such as Pumas^® which is gaining attention and slowly getting recognition.^[40,41]

We also found that recently, there has been an increase in the number of publications involving partnerships between developed and developing countries. One of the potential reasons could be the efforts that were made to promote global collaboration, such as the joint meetings of the Population Approach Group of Australia and New Zealand (PAGANZ, https://www.paganz.org/) with other population approach organizations groups such as the pharmacometrics research group in South Africa in 2002 and the group in Japan in 2006.^[42] Moreover, an international collaboration between developed countries and developing countries in scientific research is an effective way to build scientific capacity and share resources.^[43] Authors from US-based institutions were found to be most collaborative (Figure 4), probably a consequence of the large number of US-based researchers.

Funding support is an important part of research. Although majority of the 100 most cited articles were funded locally, there was a slight increase of articles that received international funding. This shows that with the availability of international funding support, countries that do not have significant national research budgets could perform highly cited population modelling studies and thus reach beyond national borders.

We also found that there were (mostly the same) top five journals that published 43-44% of all 100 most cited articles (Table 5), showing that if an article is published in one of these journals it likely has a higher probability of becoming an impactful, highly cited article. This may be valuable information for researchers who are newer in this field.

One of the limitations of this work may be the limited search involving only the Scopus database, while other conventional medicine databases, such as Web of Science, and Google Scholar, were not searched. Scopus database was selected as it provides about 20% more coverage than Web of Science, while Google Scholar can deliver results of inconsistent accuracy.^[44,45] Still, there is a possibility that some influential papers might have been overlooked. Additionally, manuscripts that did not report author-provided keywords, which were needed for the current analysis, had to be excluded as these were needed for the bibliometric analysis. Furthermore, as this study only focused on the 100 most cited articles, it should be kept in mind that the results/trends may not be representative of all published articles on population modelling. However, focusing on only these most impactful articles enabled us to confirm that each of the articles were indeed about our topic of interest, and to explore and compare the trends in more detail. One should also note that the citation count does not directly indicate the quality of an article but rather only enables a quantitative evaluation of the scientific impact of an article in a specified research field. Additionally, because citations accumulate over time, papers published earlier understandably have a higher chance of receiving more citations than those published later, making this an important consideration when using the citation count to rank individual papers; we tried to address this by calculating the number of citations per year (since an article has been published) (Online Resource 1).

We hope that our work will encourage research on topics and/or populations that are currently less represented in such impactful highly cited articles. For example, as the majority of the impactful studies described in the top 100 most cited articles were population PK studies, the field would benefit in focusing (even more) on population PD or PKPD studies, to e.g. gain knowledge about the drug effects too. Although the trends of highly cited papers appear to be slowly improving when involving the paediatric population, there is much room for research and improvement in this special population. It is likely, however, that the current most popular topics in this field will continue to gain attention and influence in the research community in the future.

We bibliometrically explored the 100 most cited articles in the population PK, PD or PKPD modelling and the research trends in this field over time. Majority of the studies described in the 100 most cited articles were published by top five journals and were population PK modelling studies looking into immunosuppressant or antibacterial drug data, involving adult population, and using NONMEM® for the analyses. There was a positive shift from studies using PK only models to PKPD models in the recent years. Apart from that no other metric that we looked at changed dramatically in the 100 most cited articles in the recent years, compared to from inception.