Universities Patent Quality Indicators (UPQI): A Bibliometric and Systematic Review

According to the European Patent Office (EPO), European patent application increased 34% of patents between 2009 and 2020. USA patents represent about 25% of European patent applications in 2021. U.S. Patent and Trademark Office received more than 646.244 patents, i.e., increase 33% since 2009. Patent applications to the European Patent Office in 2021 increased by 4.5%.^[1] Similarly, university patent applications have increased over the years. Since 1995, the number of Patent Cooperation Treaty (PCT) applications filed by universities and public research institutes has been steadily increasing. Between 1995 and 2008 the average annual growth rate in academic patent filings was 13.3%. Between 2009 and 2016 there was an average annual growth of 2.4% in PCT applications, 2.3% in university applications, and – 0.4% in public research institute applications.^[2]

The data on patent deposits may reveal important trends in the innovation process and, with respect to patenting activity, the role of universities stands out, acting as intermediaries and contributing with results in innovation activities in some specific segments and also acting in the university-business partnership. Since 1970, governments have promoted initiatives aiming to bring universities and industry closer together, to highlight their important role in the dynamics of knowledge production and in the process of technological innovation.^[3]

Examining patented technologies can indicate the direction of technological change and the most widely used. Increasingly, statistics on patents are used, in various contexts, as indicators of the outcome of invention and innovation activities, because the number of patents granted can reflect their technological dynamism and measure technological competitiveness.^[4,5] The scientific literature on the determinants and impact of innovative activity has increasingly used patent data, due to the close relationship between patents and innovation outputs, which allows the identification of changes in the structure and evolution of the inventive activity of countries, industries, firms, and technologies, allowing the mapping of shifts in technological dependence, diffusion, and penetration.^[6–8]

The quality of the innovation results can be characterized in terms of commerciality, as the ability of the patented invention to reach the market and contribute to the company’s competitive advantage; originality of the technological combination, indicating the ability of the focal company to generate new architectures and recombine knowledge; technological recognition, signaling the technical value of the patented invention recognized by third parties; and internal value, as a proxy of the private value attributed to the patented technology by its assignee.^[9]

To measure technological innovation, patent analysis is commonly adopted and can play a key role in understanding the link between scientific research or discovery and application.^[7–14] According to Narin,^[15] patent bibliometrics is “for the use of patents, and patent citations in the evaluation of technological activities.” These patent analyses usually reckon with patent counts or citation frequencies; however, these simple statistical numbers are not enough to assess the entire performance of a nation or a corporation. Therefore, mapping out a system of indicators, considering both the quantity and quality of patents and combining them to assess competitiveness enables understanding technological development, activities and innovation performance of countries.^[16–18]

The combination of bibliometric and systematic literature reviews has gained popularity in review research. It is an important tool to quantitatively analyze the evolution and performance of literature in a specific knowledge field.^[19–23]

Measuring patent performance is useful for understanding the technological development of countries, organizations and individuals: this approach is abundantly featured in the literature, assuring that the establishment of patent indicators is critical and distinctive for measuring innovation through patent analysis. However, few studies have addressed the theoretical and empirical context of selecting “appropriate indicators” and patent quality can be reflected by various indicators, whose main analysis process is to use statistical analysis, multivariate analysis, or other quantitative models to examine and interpret each field of a patent, such as application date, applicant name, applicant country, references, and international classification.^{[6,8,17,24-26]}

In this scenario, it can be realized that investigating the literature on UPQI’s, to identify, organize, summarize and to comprehend their diverse uses and scope can be useful for researchers and institutions to offer them significant guidance in the decision-making process and measurement of patent quality.

Higher-quality patents are more likely to contain technological advances that can create subsequent innovations ^[9,27,28] and patent quality is introduced to evaluate researchers’ quality and patent viewpoint. Multiple indicators are identified for patent quality analysis according to the studies’ purposes, but there is a lack of consolidation of these metrics, in a common overview, due to the complexity of the process of choosing which metric to adopt, and the challenge of providing a complete catalog of patent quality metrics, providing the opportunity for this research to contribute to summarizing and organizing this theme.^[8,9,26]

Considering the role of universities in the innovative context and the organization of patent metrics in the literature, we intend to contribute to filling this gap in the understanding of the dynamics of patent indicators whose main objective of this research is to present a bibliometric and bibliographic review of patent quality assessment indicators from the databases of Web of Science journals. We consider as specific objectives of this study: (1) Present the bibliometric data and indicators found in the selected articles and relate the patent indicators of Universities; (2) Present the relationship and grouping between articles, authors and countries and the evolution of the studied theme; (3) Identify and present the methods, quality indicators analyzed according to the area of knowledge.

Innovation is an ongoing, continuous process that plays a pivotal role in the advancement of regions and organizations, encompassing various sectors of the economy, including essential government services such as health and education.^[5,29-31]

Patents are a significant indicator of technological innovation, and a reliable measure for evaluating innovation, which is inherently characterized as a dynamic and nonlinear process. Furthermore, empirical evidence suggests the existence of historical dependence on innovation activities.^[32–36] Additionally, higher-quality patents have a higher likelihood of containing technological advances that can lead to subsequent innovations.^[29]

Patent statistics offer publicly available and regularly updated data, making them valuable sources of information, serve as formally and publicly verified metrics for measuring the outcomes of research, and provide valuable insights into the organizations that hold the patents.^[18,52]

The quality of patents in a country indicates the level of innovation and technology within that country. ^[5,14,52] Currently, there are two orientations for assessing patent quality. First, researchers choose appropriate assessment indicators based on their research objectives. For instance, the number of patent citations has been used to assess the impact of a patent on future technologies, while the number of claims or patent families has been employed to understand patent scope.^{[14,37,38,52]} Second, patent quality can be evaluated by assigning weights to different patent indicators and calculating a composite measure as the basis for assessment. Some authors’ studies introduced a novel approach by utilizing multiple indicators without subjectively assigning fixed weights.^[5,6,14] Assessing patent quality involves utilizing patents as a reliable source of information for evaluating technological development.^[6,39]

In addition, the increase in the number of patent filings by universities has in turn raised another concern, the quality of university patents and quality assessment indicators and measures available for evaluation. Several studies have addressed this question, Owen-Smith and Powell,^[40] Verspagen,^[41] Sampat, Mowery and Ziedonis,^[42] Chen and Guan,^[43] Nishimura and Okamuro,^[44] Tseng et al.,^[24] Huang et al.,^[17] Motohashi and Tomozawa,^[28] Crescenzi and Jaax,^[45] Kolympiris and Klein,^[46] Tahmooresnejad and Beaudry,^[47] Qiu and Yang,^[48] Briggs and Buehler,^[49] Schmid and Fajebe,^[50] Barrichello et al.,^[51] Sun et al.,^[52] Martinez and Sterzi^[53] but few summarize and assemble patent quality indicators. It reinforces that studying this topic can provide scientific contributions to the evaluation of the innovation process.

The establishment of patent indicators is crucial for patent analysis, yet few recent studies have thoroughly examined the theoretical and empirical foundations for selecting suitable indicators. Scholars have developed different types of patent indicators based on attribution and purpose. In our classification, these indicators are divided into three stages according to their purpose: motives (why), technological strategy (how), and value produced (what).^[24] For example,^[54] used patent counts and patent citations to study the innovative performance of international companies in high-tech industries. utilized various indicators, including the number of patents, Current Impact Index (CII), Essential Patent Index (EPI), and Essential Technological Strength (ETS), to evaluate the technological innovation competitiveness of three high-tech industries in Taiwan.^[17]

Another research determines that improving patent quality requires an understanding of the current level of quality, which relies on the ability to measure the quality of patents, addresses this issue, and contributes to the measurement of patent quality, specifically focusing on the technological and economic value of patented inventions and their potential impact on subsequent innovations, proposing a comprehensive set of indicators that capture various aspects of quality, which are often interconnected. These indicators encompass predominantly technological aspects, such as backward citations, as well as predominantly economic aspects, such as patent renewals. Additionally, there are indicators that encompass both technological and economic dimensions, such as forward citations and generality. The interpretation of quality may vary depending on the specific indicator considered, aligning either with the notion of private value or social value.^[14]

Corroborating and enhancing the relevance of the issue, recent research has shown a trend toward employing both quantitative and qualitative patent indicators to measure technological competitiveness, with the recognition that raw patent counts provide a quantitative measure while patent citations reflect the quality of patents. Furthermore, multiple patent indicators have been utilized to provide a more comprehensive measure of innovative performance.^[17,55]

The presence of technological advancements is more probable in higher-quality patents, which can lead to further innovations.^[9,27,28] Researchers’ quality and their perspectives on patents are evaluated using the concept of patent quality. Various indicators have been identified for analyzing patent quality in different studies. However, there is a lack of a unified overview that consolidates these metrics due to the complexity of selecting the appropriate metric and the difficulty of providing a comprehensive catalog of patent quality metrics. This research aims to address this gap by summarizing and organizing the theme of patent quality.^[8,9,26]

For example, Jaffe and Lerner,^[56] Mowery and Ziedonis,^[57] Lenger,^[58] Lehmann,^[59] Cao^[60] indicate the patent application and patent grants as patent indicators, Singh and Wong,^[61] Yoon and Kim,^[62] Lissoni,^[63] Cammarano et al.^[9] include forward citations, Czarnitzki and Hussinger,^[64] Sterzi,^[65] Barirani and Beaudry,^[66] Callaert,^[67] Su et al.,^[68] Chang et al.,^[8] address other indicators, such as patent renewal, patent families, backward citations, non-patent citations, and a number of Non-Patent References (NPRs) and Sun et al.^[52] approach number of inventors, licensed patents, patent claims, and patent age. Whereas establishing patent indicators is critical for patent analysis, because they are indicators of technological innovation that promote various benefits for countries, industries, universities, and institutions but few studies have addressed the theoretical and empirical context of selecting “appropriate “indicators and patent quality can be reflected by various indicators.^[8,24,26]

In recent times, there has been extensive discussion about patent quality, including its definitions, measurement approaches, and implications for innovation, entrepreneurship, and technology development. Regardless of the specific definition proposed, most stakeholders agree on the need to “raise the bar” by improving the overall quality of patents granted worldwide. Low patent quality is widely recognized as a source of uncertainty, reduced incentives for innovation, hindered technology development, and various market failures that negatively impact innovation, entrepreneurship, employment, growth, and consumer welfare.^[14,37]

To determine whether the bar needs to be raised in any given situation, it is crucial to measure the current level of patent quality. Understanding the quality of patents is a prerequisite for evaluating the necessity of improving it.^[14]

The Overview of the Research Development provides a guideline to facilitate understanding about the research process. This section outlines all the steps required to conduct this study.

This research aims to conduct a bibliometric analysis and systematic literature review on University Patent Indicators. The research questions are listed in Table 1, along with justifications and methods of analysis. The descriptive analysis provides general information about the annual productions, annual citations, and performance of countries, journals, articles, and keywords, with the intention of providing researchers with developments in related fields.

A detailed description for each subsection in the bibliometric analysis is explained below: (1) Publication analysis: Measures the contributions of authors, countries and journals in related fields by the full count method, which gives full credits on related contributions (2) Citation analysis: Examines the popularity of an article, country, journal by measuring the frequency of citations (3) Co-authorship analysis: Analyzes collaborative efforts through the number of joint publications and discovers research hotspots through the degrees of co-occurrence of keywords (4) Bibliographic coupling: Determine the relationship of publications and countries in terms of bibliographies.

An adequate and reliable scientific database is required to ensure the validity and quality of the articles obtained. Web of Science (WoS), Scopus, and Google Scholar Web are examples of this.^[69,70] WoS was selected as the central database in this study because it is reputed to be the world’s most extensive database of scientific citation and analytical information and more, it is a significant scientific tool across countries and knowledge domains, used by global scientists in different ways to answer scientific questions.^[71,72]

Selection of Relevant Research

The research had five stages: (1) Research question design, (2) Database search, (3) Selection of relevant research, (4) Analysis and synthesis; (5) Discussion, according to Figure 1, which presents the workflow of relevant literature extraction to facilitate the researcher’s understanding. In step 1, research planning was conducted, topic defined and relevance selected to fill the gap in the literature. In step 2, Database Search, the database consulted was the Web of Science – WoS. In step 3, the research questions were defined, as shown in Table 1. The research was conducted with the following search strategies: with the combined use of the descriptors “patent”, “quality”, “universities”, “metrics”, “performance”, “evaluation”, associated through the use of the Boolean operator “AND” (patent* and university quality*) and “OR” (“patent* quality” OR “patent* performance” OR “patent metrics*”) or topic: (“patent* quality” OR “patent* performance”) OR (“patent* metrics”) OR” patent* metrics*” OR “patent* metrics*” OR “patent* metrics*”) considering the presence of the keywords in “topic” (type of research), totaling 760 articles initially analyzed by the abstracts, searching for the presence of words related to the topic quality, performance, patent indicators and metrics, and some patent-related approach, in order to select those that met the selection criteria.

In terms of eligibility criteria, the specific inclusion of productions were defined: (a) of the article/review type, excluding other types of documents such as editorials, books, chapters, proceeding articles; (b) related to the topics of interest considering the categories Administration; Economics; Educational Research; Information Science; Library Science; Business; Interdisciplinary Applications of Computer Science; Computer Science Research Management; Computer Science Information Systems; Scientific Disciplines of Education; Corporate Finance; Public Administration; and Law; (c) published between 1945 and 2020.

The analysis of the articles was carried out in the following steps: (1) reading of the abstracts of the articles; (2) full reading of the selected articles to identify a relevance and meet the objectives of the work, with reference to the methods used and UPQI approached in the studies.

After refinement and establishment of the selection criteria, 401 articles resulted, which were fully read, reviewed and selected for the next stage of the research, considering the presence of UPQI, or the use of a patent quality metric, resulting in 68 articles to perform the systematic-bibliometric study, to achieve the research objectives.

From the extraction of the WoS articles, bibliographic methods were applied, performing a bibliometric and systematic analysis to address, analyze, and interpret the information needed to address the research questions. For the treatment of synonyms and similar words, it was conducted using the open source software Openrefine to group the keywords and give more robustness to the data analysis and interpretation.^[73] To generate visualizations and network maps to identify the relationships of the selected items and provide easier interpretation, VOSViewer software was used, a resource developed by Eck and Waltman^[19] widely used to build network mapping of keywords, countries, and journals.^[74]

About the network visualization depicts the co-authorship of countries, co-words, and bibliographic coupling, it is believed that network analysis provides researchers with a better graphical visualization about collaboration, co-occurrences and bibliographic coupling, where the relationships between the studied items are illustrated, allowing analysis and presentations from the size of nodes, the color of nodes and the thickness of connection lines, as well as the structure and collaboration of research and the relationships between countries and researchers.^{[19,26,74-76]}

This section provides a comprehensive analysis of the publication and citation distribution of the UPQI study for 2001 to 2020. Figure 2 shows the number of citations and publications of the UPQI within the study period. Generally, the UPQI study increased gradually over the years, with growth having its peak in 2019. Based on the findings shown in Figure 2, the publication of the UPQI in 2001 is limited to only 1 published article using the UPQI in Business and Economics. After that, it grew steadily until 2013 then increased exponentially in 2019.

Figure 1:
Research workflow for a systematic review with bibliometric analysis

This scenario indicates that the UPQI gained traction among academics in the past decades, especially in 2013, where the number of published articles was highest, with 7 articles. Although the annual scientific publication declined after 2013, it showed a continuous upward trend until 2019. Unsurprisingly, there is a quick drop in scientific production in 2021, as this study involved the bibliometric analysis until April 2021 only. Although 2021 is unfinished, there are few articles published and indexed in the WoS.

It is a usual phenomenon to vary over time, and the citation of UPQI has fluctuated over the years, mainly due to the impact of consolidated and benchmark research at certain points in time, developed by authors Owen-Smith and Powell (2003); Sampat, Mowery and Ziedonis (2003) and further developed by authors Tseng, Hsieh, Peng, Chu (2011), Sterzi (2013), Subramanian, Lim and Soh (2013), Arts, Cassiman and Gomez (2017), Kolympiris and Klein (2017). In 2003 recorded the highest citation, at 366, as many researchers referred to previous studies as a reference grew between the years 2002 and 2006. After that, the years 2011, 2013, and 2017 were well-referenced. The lowest citation occurred in 2021 because the data collection period ends on April 2021.

Note. TP indicates the complete publication of articles according to countries, TC is the total citation, while AC is the ratio of total citation per total publication and TLS is total link strength.

A total of 23 countries contributed to the study of the UPQI. The most productive countries are listed in Table 2. Most of the research work is concentrated within the USA, with a total of 14 publications. China emerged as the country with the second-highest number of publications (11), followed by

Figure 2:
a and b: Annual scientific production – Citation and Publication (2001–2021)
Note. The left figure is (a) citation and right figure is (b) annual publication.

Italy (7) and Spain (7). Publications from the USA recorded the highest number of citations (914), followed by Norway (222), Netherlands (205), China (165) and England (135). The total link strength attributes are essential indicators of both the quantity and strength of connections between items. In the case of co-authorship links among researchers, the Links attribute denotes the number of co-authorship links a given researcher has with others. Meanwhile, the Total link strength attribute represents the combined strength of the co-authorship links that a researcher maintains with their collaborators.^[19]

Expanding upon the research analysis, specifically focusing on Total Link Strength (TLS) citation, the United States (62) exhibits articles associated with 18 different countries in Table 2. In comparison, the Netherlands (38) demonstrates articles linked to 13 countries, Spain (28) with 10 countries, France (23) with 11 countries, and Italy (19) with 10 countries. It is noteworthy that China (15) displays a lower TLS citation count, having connections with 9 countries in terms of citations, but a higher TLS when assessed through coauthorship analysis.

Examining Table 2, it becomes apparent that China distinguishes itself in terms of co-authorship (8), along with the United States (6), the Netherlands (6), and Belgium (6). This observation is particularly significant as it demonstrates a higher level of collaboration, considering the quantity of papers, thereby reinforcing the existence of joint research efforts.

Academic journals are platforms used by scholars to share new insights and knowledge. Therefore, it is vital to undergo the analysis of the most fruitful journal in the field of Patent Quality Indicators. The most productive journals can publish high-quality peer-reviewed articles, disseminating the latest developments in related fields by conducting rigorous peer reviews on articles.^[77] In this study, 68 journal articles were published in 42 journals. Table 3 shows the top 5 most influential journals in the area of the UPQI, ranked according to the number of publications. There are 28 articles published in the 5 journals, accounting for 24.35% among all publications. Scientometrics has published the most articles, 8 publications, followed by Research Policy (7) and Journal of Technology Transfer (6).

The number of citations for each journal is presented and shown in Table 3, along with the average citations. For example, the publications from the Research Policy received 667 citations and ranked first in terms of citations, followed by Journal of Technology Transfer (192) and Scientometrics (150). In terms of average citations, the top 3 journals with the highest average citations were Research Policy (95.29), Technological Forecasting and Social Change (43.67), and Journal of Technology Transfer (32).

In the context of the impact factor, Technological Forecasting and Social Change reported the highest impact factor, 10.884. The journal with the second-highest impact factor is the Research Policy (9.437), followed by the Journal of Technology Transfer (5.337).

The dedicated IPR journals did not appear among the most prominent, but are listed with smaller number of publications, between 1 and 2, such as, Technovation (2), International Journal of Technology Management (2), International Journal of Innovation Science (1), World Patent Information (1), indicating their contribution to research. Furthermore, it is possible that due to the broad scope and relevance of its applicability in various domains, the study may have encompassed other significant journals that are not specifically focused on the subject matter.

Table 8 presents the main cited articles involving the UPQI in the investigated period, Authors, Journals, Year of Publication, Total Citations, Average Citations, Type of Methodology and Technique used. Total citation plays an essential role in determining the impact of an article, because a higher number of citations correlates with the quality of the published paper.^{[22,69,78,79]} Among 68 articles, the article with the highest citations is “Academic patent quality and quantity before and after the Bayh-Dole act in the United States” by Mowery and Ziedonis,^[57] published in the Research Policy, and had a total of 257 citations. Concurrently, the “Long memory and regime-switching” the highest average citation is (12.24). The “Initiatives to promote commercialization of university knowledge” by Rasmussen et al.^[77] ranked as the second-highest cited article (222), with an average citation of 13.06, followed by Owen-Smith and Powell,^[40] which is “The expanding role of university patenting in the life sciences: assessing the importance of experience and connectivity” (221). Almost all papers use Quantitative Methods, with 54% of those employing regression analysis: the sole exception, “The state-of-the-art on Intellectual Property Analytics (IPA): A literature review on artificial intelligence, machine learning and deep learning methods for analyzing Intellectual Property (IP) data” used a qualitative approach.

About the network visualization portrays the co-authorship of countries, co-words, and bibliographic coupling, it is believed that network analysis provides researchers with a better graphical visualization about collaboration, co-occurrences, and bibliographic coupling, where the relationships between the studied items are illustrated, allowing analysis and presentations from the size of nodes, the color of nodes, and the thickness of connection lines, as well as the structure and collaboration of research and relationships between countries and researc her.^{[19,26,74-76]} In this study, the minimum paper of one country was set to four to facilitate network analysis in most collaborating countries publishing surveys of the University Patent Quality Indicators. This means that all the selected countries should have at least four publications.

The top collaborative countries, ranked according to the total link strength, are presented in Table 2. The total link strength is defined as the total number of articles published by authors through collaboration among different countries. In other words, the most collaborative country often shows high values in total link strength.

The USA is the leading country with the highest publication (14) in the area of the Universities Patent Quality Indicators and China is the country with the second-highest number of publications (11) and respectively total link strength (5) and (8). Generally, in the analysis of country co-authorship, it can be seen that developed countries (i.e., USA, England, and France) are interested in collaborating with other countries in publishing articles on UPQI relative to developing countries such as Taiwan and Singapore.

Figure 3 shows the visualization map of countries co-authorship in publishing Universities Patent Quality Indicators switching literature with different node sizes and colors. The node size refers to the number of published documents of a country, where the bigger the node, the higher the documents published from the country. These countries were categorized into seven main clusters, presented in seven colors: red, green, blue, dark yellow, violet, light blue, and orange. The details of the main five clusters with the most sub items are listed below.

Based on the node sizes shown in Figure 3, one can see grouping and proximities in relation to the co-authoring countries, where France and Italy, Germany and México, China and the USA represent closeness in terms of publications in the field of UPQI. China and the USA are the countries with the most cross-country research, with the Chinese broadening their horizons the most, with research with England, Taiwan, Germany, France, Spain and the Netherlands. A country that has a strong TLS is Spain, with research with Mexico, China, France, Belgium and Canada. The USA also has a stronger relationship with China, England and France.

Figure 3:
A network analysis of countries co-authorship based on document weights

Keyword co-occurrence analysis is a significant tool used in bibliometric analysis as it helps reveal the central theme of the research because it indicates a relation between two concepts.^[19,74,80] This analysis was conducted using 451 keywords to reveal the predominant research points or themes in the UPQI study. In order to identify the co-occurrence of essential keywords, the minimum frequency of keywords was limited to three, which resulted in 5 keywords used in the visualization analysis. Table 4 highlights the co-occurrence of the top 10 keywords, organized according to total link strength from 11 appearances. The most frequent keywords with Occurrences and TLS are “Innovation” (26)(91), followed by “Patent Quality” (18)(60), “Performance” (18)(62), “Technology” (16)(61) and “Bayh-dole Act” (14)(46). The co-occurrence of the authors’ keywords can be visualized by mapping the network, as shown in Figure 4. The sizes and colors of the nodes play different roles in the co-occurrence analysis, where the sizes reflect the frequency of the authors’ keywords in the University Patent Quality Indicators literature, while the colors represent the number of clusters. Highlights include “Innovation”, “Performance”, and “Patent Quality” as the most relevant keyword linked to “Patent”, “Performance”, “Quality”,

“Collaboration”. “Patent Quality” linked to “Technology Transfer”, “Commercialization” and “Bayh-dole act”. Regarding Overlay, keywords such as “Academic entrepreneurship”, “Quality”, “Research and Development” are more recent, followed by “Commercialization”, “Science”, “Technology transfer”. Finally, “Citations”, “Indicators”, “Collaboration” appeared the longest.

Furthermore, in the blue cluster there is a relationship between invention, innovation, performance, citations and quality, networks and collaboration, that is, there is a quality indicator related to the performance of innovations and inventions and in biotechnology. In the green cluster is a relationship between investment in R&D, science, spillovers and indicators. It is known that indicators can measure the return on research and development. In the yellow cluster are related impact, patent quality, technology transfer, commercialization and academic entrepreneurship, that is, entrepreneurship is able to generate impact in the generation of patents, quality products and services, which can be transferred and commercialized.

Kessler^[81] first introduced bibliographic coupling to identify the similarities shared by two articles. Bibliographic coupling measures the subject similarity of the documents analyzed and seeks to determine the relationship between them.^[82]

Selecting authors with at least 20 citations, in order to obtain understand the authors’ edges and vertices, eight clusters with 4.806 citations were categorized in Figure 5, where the red cluster is the largest, comprising 27 authors and a total of 3.040 citations, with an average of 112.59 citations per article. Most publications in this cluster work on university quality and patenting, the relationship between scientific publications and patents, the impact of Bayh-Dole acts on the quantity and quality of patents. The most cited publications in this cluster (red) focus on comparing the impact of the Bayh-Dole law on university patents and universities’ knowledge commercialization initiatives (Mowery and Ziedonis;^[57] 403 citations; Rasmussen et al.;^[77] 222 citations).

The yellow cluster includes 5 authors with 168 citations, with an average of 33.6 citations per publication. The authors in this cluster studied the impact of collaboration and networks on patent quality indicators, patent co-citation networks, the impact of scientific networks and university-firm contracts on university patenting, and biotechnology patent mapping. The most cited authors in this cluster investigated the impacts that collaboration and networks have on patent quality, from the reality of nanotechnology innovations in Canada. (Beaudry and Schiffauerova;^[83] 84 citations) According to Figure 5, these authors represent the strongest and most robust relationship in the bibliographic coupling analysis, given the amount of links to other authors.

The dark blue cluster contains 6 authors with 418 citations, with an average of 69.7 citations per authors. The authors in this cluster

Figure 4:
A network analysis of keywords co-occurrence

Figure 5:
Bibliographic coupling of publications

investigated the issue of patent partnerships and relationships, data analysis to identify evolution, and patent strategies. The most cited authors in this cluster studied patent data to analyze distribution and technology strategies in the amorphous silicon thin film solar cell industry (Tseng et al.;^[24] 88 citations).

The green cluster contains 8 authors with 386 citations, an average of 48.25 citations per author. The authors in this cluster analyzed patent similarity using text matching and patent citations to calculate science linkage and technological similarity between patents. (Arts et al.;^[84] 62 citations)

The purple cluster has 6 authors. This cluster recorded 252 citations, with an average of 42 citations per article. In this cluster, publications focus on themes of intellectual property, entrepreneurship, university research practices and patents, and the relationship between universities and industry, bringing the theme of the Triple Helix. The most cited authors in this cluster address the state-of-the-art on Intellectual Property Analytics using artificial intelligence techniques through literature review. (Aristodemou and Tietze;^[85] 52 citations)

Figure 6:
Bibliographic coupling of countries overlay Visualization

The light blue cluster features 5 authors, with 196 citations, an average of 39.2 citations per author, and brings the discussion between publishing and patenting scientists and their role in patenting performance as “bridging scientists.” (Subramanian et al.;^[86] 52 citations)

The brown cluster is composed by 4 authors, sums 186 citations with an average of 46.5 citations per author, and studies industrial clusters in Japan and their conditions for these projects to be successful. (Nishimura and Okamuro;^[44] 64 citations) Finally, orange cluster counts 4 Italian authors, with 160 citations and 40 citations on average per author. These authors research collaborative behaviors between firms, research groups and universities considering R&D alliances inside the innovation process. (Cammarano et al.;^[9] 40 citations)

Generally, these clusters are similar in research areas, however, the authors focused on different issues. Nevertheless, the analysis of patent data, use of indicators was part of the studies within the

Bibliographic coupling of countries attempts to calculate the similarity between pairs of countries through citations, showing how often these countries share similar bibliographies. The bibliographic coupling visualization map can be presented in various patterns of colors and node sizes. The colors reflect the number of clusters present in this study, while the node sizes represent the country contributions, where the larger the node, the more significant the country contribution.^[69,87,88]

The coupling analysis revealed five groups where the USA, China, Italy, Spain, the Netherlands and Canada are the leading countries in studies involving the University Patent Quality Indicators with the greatest scope among the countries. Taking a temporal analysis, research from Australia, Denmark, USA, Japan, Netherlands, Norway, Taiwan and Turkey are incipient, and work from China, Germany, Spain, France and South Korea and Brazil, England, Italy and Mexico are more contemporary, as seen in Figure 6.

Subsequent to conducting the bibliometric analysis, a rigorous systematic literature review was then conducted to determine the application of the UPQIs by analyzing within the research areas, the articles’ objectives, outcomes, and patent quality indicators.

Table 5 describes the most highlighted research areas by remarks and percentages: Business and Economics (123, 41.98%), Engineering (42, 14.33%), Computer Science (35, 11.95%), Information Science and Library Science (28, 9.56%). The most referenced indicators in the literature are Patent application (81, 27.60%), Patent granted (52, 17.70%), Forward citations (53, 18.10%), Family size (19, 6.50%). Comparing research areas and indicators of patent quality, it can be observed that in Business and Economics, Patent application (39), Forward citation (22), Patent granted (22) and Backward citation (9) are the most studied in the literature. In the Engineering area, Patent application (11), Forward citation (8), Patent granted (6) show up more prominently. Also stands out Computer Science, using the indicators Patent application (8), Forward citation (6), Patent Granted (5). Information Science and Library Science using the indicator Patent granted (7) and Operations Research and Management Science, with the indicator Forward citations (6). Analyzing the extent of the indicators, we observe the presence of Patent application, Patent granted and Articles published (Research) in most of the research areas, indicating the relationship between research and development and innovation at Universities.

On Table 6, relating techniques applied and indicators of quality of patents, it is evident that most of them use regression (65.79%) and descriptive works (9.47%). As for the indicators, the most relevant are Patent Application (25.79%), Forward citations (19.47%) and Patent granted (19.47%). In the intersection is the concentration of the indicators Patent Application, Patent granted, Forward citations, Backward citations mainly using regression, and the indicators Patent Application, Forward citations and Patent granted are the most overarching, present in most of the analyses.

Analyzing Table 7, besides the evidenced of the indicators most commonly used and cited in the literature over the time, Patent application (50, 19.16%), Patent granted (37, 14.18%), Forward

citations (37, 14.18%), Patent claim (11, 4.21%,), Family Size (11, 4.21%,), Backward citations (11, 4.21%,), it can also be observed that the indicators Patent application, Patent granted, Forward citations, Backward citations and Patent Licensed are among the first used by researchers. And the years 2013 and 2019 are the years with the highest number of papers with use of indicators.

The selected articles with the most citations in the research area are discussed in this section. The use of the average citation ensures the equality and fairness of this study (Table 8). As for methodology, 87% of the papers have a quantitative approach, 92.30% of the most cited papers are quantitative and apply regression as a technique. Descriptive papers are also present among the most cited papers. In Table 8, highlights the coverage of the indicators Patent Applications, Patent Granted, Forward citations, Backward citations, Patent Licensed, Number of spin-off companies, Patent claims, Science Linkage (SL), Science Strength (SS), Current Impact Index (CII), Patent Portfolio, Technology strength, Family Size, Patent Portfolio, Number of Non-Patent References (NPRs), Patent assignee, Patent-to-Paper Citations, Patent disputes, Originality, Experience, Number of applicants, Number of inventors, Co-publication and co-invention, Patent Applications, Number of citations present in the most used papers in the literature.

This research theme covers a large portion of all research works (41.56%). Mowery and Ziedonis^[57] summarize the results of empirical analyses of data on the characteristics of the pre-and post-1980 patents of three leading US academic patenters—the University of California, Stanford University, and Columbia University and the most significant change in the content of research at these universities, one associated with increased patenting and licensing at both universities before and after 1980, was the rise of biomedical research and inventive activity, but Bayh–Dole had little to do with this growth. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Forward citations, Backward citations and Number of citations.

Owen-Smith and Powell^[40] have a proposal to extend debates about the sources of university capabilities at research commercialization, drawing a quantitative data for a panel of 89 research-intensive US universities, modeling the relationship between technology transfer experience, embeddedness in biotechnology industry networks, basic science quality and capacity, and citation impact measures of university life science patents. Through patent citations and industry connections, it was possible to observe improvement in the development of

patent portfolios, i.e., the greater the number of citations, the more technologies available in the portfolio, which is enhanced by industry connections. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Patent claims, Forward citations, Number of citations.

Sampat et al.^[42] analyze the impact of the Bayh-Dole Act of patent citations before and after the Bayh-Dole Act. Results indicate no decline in the “quality” of university patents during the 1980s. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Patent Licensed, Forward citations.

Jaffe and Lerner^[56] examine the initiatives since 1980 to encourage patenting and technology transfer at the national laboratories, patent policy and federally funded R&D. The analyses suggest that the policy reforms of the 1980s had a positive effect on technology commercialization as well as patenting, although within laboratories there is some evidence of an association between increased patenting and a decline in patent quality, the overall increase in patenting does not appear to have been associated with an overall decrease in quality, as was the case for universities. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Patent Licensed.

Verspagen^[41] intends to give an overview of the debates surrounding the introduction of the Bayh-Dole Act in the USA, and the possible adoption of similar legislation in Europe and has the ambition to provide an overview of the academic literature in economics dealing with university patenting, and to draw policy lessons from this. also discusses Bayh-Dole Effects through an Empirical Evaluation and although the results are mixed, the evidence that patents actually facilitate knowledge transfer is mixed, i.e., more case studies are required. The article cites and uses in your proposal the indicators: Patent Granted.

Arts et al.^[85] use a text mining technique based on common keywords to develop a new measure of technological similarity for all granted utility patents (USPTO) between 1976 and 2013 and illustrate how a text mining tool can be used to more accurately measure technological similarity between patents on a continuous scale. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Number of applicants, Number of inventors, Number of assignees, Family Size, Patent Portfolio.

Kolympiris and Klein^[46] analyze the impact of academic incubators on the quality of innovations produced by US research-intensive academic institutions. The results show that the establishment of a university-affiliated incubator is followed by a reduction in the quality of university innovations, even when controlling for the endogeneity of the decision to establish an incubator using the presence of incubators at peer institutions as an instrument. University incubators compete for resources with technology transfer offices and other campus programs and activities, so the useful results they generate may be partially offset by reductions in innovation elsewhere. The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Forward citations, Non-patent citations, Number of Non-Patent References (NPRs).

Sterzi^[52] intends to contribute to this debate by analyzing the quality determinants of a sample of UK academic patents and, in particular, by investigating whether the ownership structure is to some extent correlated with patent quality. And aims to assess the extent to which patent ownership and quality are correlated. the

results suggest a higher quality for academic patents owned by commercial companies in the short to medium term with respect to academic patents owned by universities The article cites and uses in your proposal the indicators: Patent Applications, Patent Granted, Patent claims, Patent assignee, Forward citations, Backward citations, Non-patent citations, number of Non-Patent References (NPRs), Patent-to-Paper Citations, Patent disputes, Originality, Experience.

Subramanian et al.^[87] examine heterogeneity within the firm’s scientific human capital, emphasizing the distinctive role of scientists as a bridge who engage in patenting and publishing. The results suggest that each profile plays a different role and reduces the firm’s dependence on external scientific expertise derived from university partnerships and bridging scientists play a complementary role in university-industry collaboration. The article cites and uses in your proposal the indicators: Number of applicants, Number of inventors, Forward citations, Science linkage, Co-publication and co-invention.

Rasmussen et al.^[83] applied case studies of four European universities of science and technology in Finland, Ireland, Norway and Sweden, this article analyzes several commercialization initiatives that support mechanisms for entrepreneurship. The article cites in your proposal the indicators: Patent Applications, Patent Granted, Patent Licensed, Number of spin-off companies.

Tseng et al.^[24] employed a set of indicators to analyze the technological development of a-Si TFSC, using patent portfolio, through CHI Research patent indicators combined with the theoretical review to categorize corporations according to their technical competence in order to understand their patent technology and found that the main technological field has reached the maturity stage in the technological life cycle of this product; in addition, four strategic patent clusters were identified, concluding that the number of patents can indicate the overall situation and trends in a certain field and that the patent indicator is an important tool when exploring technological development, although patents are always less evident than those related to technological development because the patent review period is very long. This paper provided a great contribution to the studies due to the structure and metrics addressed. The article cites and uses in your proposal the indicators: Patent applications, Patent Granted, Forward citations, Science linkage, Science Strength (SS), Current Impact Index (CII), Patent Portfolio, Technology strength.

Nishimura and Okamuro^[44] examine the effects of the ‘‘Industrial Cluster Project’’ (ICP) in Japan on the R&D productivity of participants, using a unique dataset of 229 small firms, and discusses the conditions necessary for the effective organization of cluster policies. The results suggest that participation in the cluster project alone does not affect R&D productivity. Moreover, research collaboration with a partner in the same cluster region decreases R&D productivity both in terms of the quantity and quality of patents. This article cites and uses in your proposal the indicators: Patent Applications, Patent claims, Forward citations.

Aristodemou and Tietze^[86] examine the effects of the “Industrial Cluster Project” (ICP) in Japan on participants’ R&D productivity, using a unique dataset of 229 small firms, and discusses the conditions necessary for effective cluster policy organization. The results suggest that participation in the cluster project alone does not affect R&D productivity. Moreover, research collaboration with a partner in the same cluster region decreases R&D productivity, both in terms of patent quantity and quality. Although, unlike the previous projects, ICP aims to promote the local innovation network, including collaboration with major national universities within each cluster and that local firms that collaborate with partners outside the cluster show higher R&D productivity, both in terms of quantity and quality. The article cites and uses in your proposal the indicators: Patent applications, Number of citations.

The bibliometric analysis revealed significant research from University Patent Quality Indicators (UPQI) with a continuous ascending distribution. Meanwhile, the USA is the leading contributor, reporting the highest publications, and subsequently Scientometrics was ranked as the leading journal, but the International Journal of Forecasting was the journal with potential in the near future. The most influential articles on business and economics were written by Mowery and Ziedonis,^[57] Rasmussen et al.,^[77] and Owen-Smith and Powell,^[30] in I want respect as citations. In terms of number of indicators, Tseng et al.,^[24] Sterzi,^[65] Arts et al.,^[85] Subramanian et al.^[87] were the most collaborative in surveying patent quality indicators. As for the collaboration network, countries such as France, Italy, Germany, the Netherlands, China, and the USA were found to be the most influential in terms of publications in the field of UPQI. The most common keywords are Innovation, Bayh-Dole Act, Performance, Technology, Patent Quality, Research and Development, corroborating with the analyzed works and reconciling, as for example, with Tseng et al.^[24] who state that the number of patents may indicates the overall situation and trends in a given field and that patent indicators are an important tool when exploring technological development. From the coupling of countries, the US has the most substantial influence in the area of UPQI, as some countries are coupled to the US and recently, Italy, England and France are coupled. From the coupling of articles, Mowery and Ziedonis,^[57] Sampat, Mowery and Ziedonis,^[42] Beaudry and Kananian (2013), Cammarano et al.,^[9] Briggs and Buehler^[49] have the most substantial influence in all the publications.

Patent filings have exhibited a global increase, paralleled by the growing number of patents registered by universities in recent decades. This trend has sparked academic interest, resulting in numerous studies adopting diverse approaches to investigate the phenomenon.

In addition to their role in safeguarding intellectual property rights and serving as repositories of technological knowledge, patents are utilized as indicators to evaluate the technological innovation and performance of countries, organizations, and researchers. However, these evaluations have predominantly focused on the quantity of patent applications or granted patents, with little attention given to patent quality.

Assessing patent quality is a multifaceted undertaking due to the broad spectrum of protected technologies and the wide range of applicability and impacts they can generate. Despite the inherent complexity, certain studies have endeavored to address patent quality by employing more objective and quantifiable indicators such as citations, filings, grants, licensing activities, among others.

Nevertheless, existing research on the subject generally concentrates on individual or a limited set of quality indicators, resulting in a fragmented and dispersed literature. This fragmentation hampers the ability of science, technology, and innovation managers, as well as policy makers, to obtain a comprehensive overview of these indicators.

In view of this backdrop, the present study aimed to organize and systematize metrics for assessing patent quality in the context of universities from Web of Science journal databases. This study brings contributions to map these patent quality indicators in the literature, identify and present the methods and techniques to evaluate the quality of patents, according to the area of knowledge. In addition, it seeks to indicate paths and solutions for researchers and institutions to evaluate patent quality, not only in universities, but also in other contexts.

The paper presents significant insights into the UPQI study, indicating that countries with a long-standing presence in research and development continue to exert influence in the field. Moreover, there is a noticeable emergence of new European, Asian, and Brazilian countries. The prominence of journals related to policy, management, innovation, and technology underscores the relevance of the topic. Additionally, the study examines the role of universities in generating innovations and patents, as well as their implications for quality evaluation and performance assessment.

While certain metrics like patent deposits, grants, and citations are commonly highlighted in academic research due to their accessibility and availability in databases, a closer analysis reveals that other quality indicators offer fresh perspectives and more reliable assessments of patents. These indicators, predominantly quantitative in nature, are more suitable depending on the research objectives and characteristics. They include licensing, the number of spin-off companies, science linkage, co-publication, and co-invention.

Furthermore, a novel approach involves combining indicators to create new measures, potentially in the form of indices. This approach provides more appropriate and diverse perspectives on the assessment of quality. The findings of the study present a well-organized set of metrics and information that can be utilized by managers, researchers, and funding agencies to inform policies and decision-making processes, thereby fostering technological development and facilitating collaborations with the industry sector.

Listing these indicators simplifies the process of searching, identifying, and deciding the use of patent metrics, as well as guides the quality assessment process based on whatever the literature has historically judged to be relevant. In addition, mapping how the process is conducted, through available methods, profile of research areas, relationships between countries and researchers, which techniques are commonly practiced in the literature over time, identifying the most relevant authors and the most representative journals on the studied topic can offer important subsidies to researchers and patent quality assessments.

Given the role and relevance of Academia in the field of research and development, its contribution in the innovation process, and the existence of limited studies that approach the context of selecting patent quality indicators, the synthesis, organization, examination and interpretation of UPQI’s may be useful for technological development because it provides significant guidance in the decision-making process and measurement of patent quality.

Additionally, the identification of the highest quality patents, those most susceptible to produce technological advances creating subsequent innovations, summarization brings contribution for professionals and managers, providing subsidies to track and evaluate the performance of what has been produced in the environment of universities, which can help in the establishment of policies, decision-making models, prioritization of investments. The lack of uniformity in the process of using metrics reveals the diversity of metrics available to indicate patent quality and also indicates the importance of summarizing these metrics and then organizing and categorizing them, which is a suggestion for future research.

This study has some limitations, the first being the accuracy of the datasets extracted from WoS. The retrieved datasets may be slightly different when performing the search on different dates, although the methodology, search steps, and keywords were presented, as WoS updates its list of published articles daily.

Second, the accuracy of the results also depends on the types of sources used to extract datasets. In this study, the datasets were extracted from the WoS database, and studies of University Patent Quality Indicators not indexed by WoS may not have been included. Therefore, other databases can be used and incorporated with WoS to obtain a large amount of literature on the topic to increase the reliability of the results.

This study was financed in part by the CNPQ Brazil. Douglas O. Cardoso acknowledges the financial support by the Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia, FCT) through grant UIDB/05567/2020, and by the European Social Fund and programs Centro 2020 and Portugal 2020 through project CENTRO-04-3559-FSE-000158.