1. bookAHEAD OF PRINT
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English
access type Open Access

New Indicators of the Technological Impact of Scientific Production

Published Online: 24 Jun 2021
Page range: -
Received: 05 Mar 2021
Accepted: 03 Jun 2021
Journal Details
License
Format
Journal
First Published
30 Mar 2017
Publication timeframe
4 times per year
Languages
English
AbstractPurpose

Building upon pioneering work by Francis Narin and others, a new methodological approach to assessing the technological impact of scientific research is presented.

Design/methodology/approach

It is based on the analysis of citations made in patent families included in the PATSTAT database that is to scientific papers indexed in Scopus.

Findings

An advanced citation matching procedure is applied to the data in order to construct two indicators of technological impact: on the citing (patent) side, the country/region in which protection is sought and a patent family's propensity to cite scientific papers are taken into account, and on the cited (paper) side, a relative citation rate is defined for patent citations to papers that is similar to the scientific paper-to-paper citation rate in classical bibliometrics.

Research limitations

The results are limited by the available data, in our case Scopus and PATSTAT, and especially by the lack of standardization of references in patents. This required a matching procedure that is neither trivial nor exact.

Practical implications

Results at the country/region, document type, and publication age levels are presented. The country/region-level results in particular reveal features that have remained hidden in analyses of straight counts. Especially notable is that the rankings of some Asian countries/regions move upwards when the proposed normalized indicator of technological impact is applied as against the case with straight counts of patent citations to those countries/regions’ published papers.

Originality/value

In our opinion, the level of sophistication of the indicators proposed in the current paper is unparalleled in the scientific literature, and provides a solid basis for the assessment of the technological impact of scientific research in countries/regions and institutions.

Keywords

Introduction

One of the most important topics in quantitative science and technology studies is the relationship between the two fields. Citation analysis is a key to gaining insight into this relationship. Citations in scientific papers to other scientific papers have long been used to assess the contribution made by individual scholars and research institutions to scientific progress, as well as to analyse the cognitive structure of science and the collaboration and flows of knowledge among scientific workers (Price, 1963). Citations made in patents to the scientific literature may similarly be used to study the influence of scientific and scholarly work on technological development. Pioneering work on patent citation analysis was conducted by Francis Narin and co-workers, who started exploring the use of patent citations to measure technological impact in the 1980s (Narin & Olivastro, 1992; Narin, Hamilton, & Olivastro, 1997). This is the perspective taken in the present communication. Just as a citation from the scientific literature is used to determine the scientific impact of a published scientific article, a citation from a patent can be used to determine an article's technological impact. The relationship between scientific citation and patent citation is an ongoing subject of study (Ahmadpoor & Jones, 2017; Poege et al., 2019; Veugelers & Wang, 2019), although some concerns have been raised about its real relevance (Bryan, Ozcan, & Sampat, 2020; Callaert, Pellens, & Van Looy, 2014). Nevertheless, patent analysis to construct science and technology indicators has become an active and complex discipline in its own right. Comprehensive overviews of the main methodological issues involved are given in Hinze & Schmoch (2004), Van Raan (2017), and more recently in Schmoch & Kahn (2019) and Veugelers & Wang (2019).

Technological research and development come at important costs. For this reason, when the development of a product has been completed successfully, it needs to be patented to obtain the protection that can allow its economic exploitation. A patent is a document describing a product, with the pertinent bibliography and citations to other patents and other documents. One part of the bibliography is provided by the authors and another part by the patent office's examiners. The evaluation procedure varies significantly from one national patent office to another, and consequently so may the bibliography, both the part required from the authors and the part incorporated by the examiners.

A patent family can be defined as a set of patents filed in various countries to protect a single invention. The protection in the country in which the first application is made—the priority country—is then extended to other countries. In other words, a patent family relates to the same invention disclosed by one or more common inventors and patented in more than one country. A first major problem in patent citation analysis addressed in the present communication is that there may be substantial differences between members of the same patent family regarding the non-patent literature (NPL) references they may contain. This is especially the case for the examiner-incorporated references since the patents of a family tend to pass different examination processes in the different patent offices. The process may be slower or faster, and may lead to more or fewer NPL references being incorporated.

The process of registering a patent to protect an invention is neither easy nor cheap. A second problem addressed in this communication is that the protection of an invention in a technologically advanced country will give it a greater competitive edge and a greater potential market than in a developing country. Seeking protection in advanced countries with larger markets is harder and more expensive than it is in developing countries with small markets because greater benefits are expected from the invention. Therefore, a triadic patent—i.e. a set of corresponding patents filed in the European Patent Office (EPO), the United States Patent and Trademark Office (USPTO), and the Japan Patent Office (JPO) for the same invention with the same applicant or inventor—tends to have more economic and technological significance than a patent family covering one or more developing countries/regions.

A third problem is that there appear to exist large differences in the number of citations to scientific papers among patents and patent families. In a citation analysis of the scientific literature, advanced citation impact indicators correct for differences in the length of reference lists in scientific papers in a subject field. A similar “citing or source” normalization would be needed in an analysis of patent citations to take different patent families’ propensity to cite scientific literature into account.

The aim of this study was to explore new indicators of the technological impact of recent research papers—indicators based on citations to the scientific literature that are found in patents, and taking into account the three problems outlined above:

To avoid differences in references to the scientific literature among patents from the same patent family, the patent family rather than an individual patent is taken as the basic unit on the “citing side” of the citation analysis. The SCImago Research Group has retrospectively assigned all non-patent references in the various members of a patent family to each patent in that family.

To differentiate among patent families according to the country/region in which patent protection was sought, a weight is assigned to each country/region based on that country/region's state of economic development. A citation from a triadic patent then has greater weight than a patent citation from an economically developing country/region. In a country/region with greater economic and technological activity, it is more difficult and expensive to obtain a patent. When an applicant selects a country/region of this type it is because the patent involves state-of-the-art technology and is expected to yield significant economic benefits that will compensate for the investment.

The reciprocal of the number of references to scientific papers in a patent family is used as a weighting factor normalizing the number of patent family citations received by a scientific paper, thus correcting for differences among patent families in their propensity to cite the scientific literature. In this way, each patent family distributes its recognition (which is already weighted by the state of economic development of the countries/regions in which its patents are applying for protection) equitably among all of its references to scientific papers.

The newly proposed indicators of technological impact are applicable to the scientific production of countries, institutions, research groups, or even individual authors.

Data

Elsevier created and maintains Scopus, a bibliographic database of scientific literature (Hane, 2004; Pickering, 2004). It indexes more than 31,000 scientific journals, conference proceedings, and books. Several works have described its characteristics (Archambault et al., 2009; Leydesdorff, Moya Anegón, & Guerrero-Bote, 2010; Moya-Anegón et al., 2007), and it has been used in numerous scientometric studies (Gorraiz, Gumpenberger, & Wieland, 2011; Guerrero-Bote & Moya-Anegón, 2015; Jacsó, 2011; Moya-Anegón et al., 2018). Scopus classifies publications by Subject Area, and then by Specific Subject Area or Category. There are more than 300 categories grouped under 26 subject areas. In addition, there is a Multidisciplinary Area that contains such journals as Nature or Science. The present study covers documents indexed in Scopus up to 2018 and published after 2002.

In 2008, at the behest of the patent statistics working group led by the Organization for Economic Co-operation and Development (OECD), the EPO released PATSTAT (“EPO Worldwide PATent STATistical Database”), designed to assist in statistical research on patents. There are other patent databases, such as NBER (the United States) and IIP (Japan). But PATSTAT has worldwide coverage, includes more information, and has some auxiliary products that can resolve various problems. For these reasons, PATSTAT has become a de facto standard (Kang & Tarasconi, 2016). Nevertheless, it has two biases. One is towards European data since national patent offices exchange information with the EPO based on formal agreements (although these agreements change over time and leave some gaps). The other is towards the patent examination process since data that are not actually required in this process may be of lower quality. For the present study, we used the 2019 Spring edition.

PATSTAT collects references both to NPL and to patents. Due to its patent application and examination orientation, its data require debugging and standardizing to allow linkage with data taken from other databases. For example, the lack of standardization has long been a cause of problems in relating applicants and inventors with the data available in company databases. The first attempts to normalize names were made with the standardization tables of Thomson Scientific's Derwent World Patent Index (2007) and the USPTO's CONAME file, and further development and improvements are continuing to be made (Coffano & Tarasconi, 2014; Lissoni, 2012; Lotti & Marin, 2013; Magerman, Van Looy, & Song, 2006; Maraut & Martínez, 2014; Raffo & Lhuillery, 2009; Thoma & Torrisi, 2007; Schoen, Heinisch, & Buenstorf, 2014).

In order to link the NPL citations in patents with scientific publications, one faces the same problem as in the case of the names of the applicants or inventors—namely, the absence of standardization. We only know of one publicly available publication that has taken on this issue. It was part of the development of Lens Influence Mapping (Jefferson et al., 2018), and used PubMed and Crossref as the scientific literature databases. However, this study indicated neither how the cases in which a patent citation could be linked to more than one DOI (Digital Object Identifier) were resolved, nor how the method ensured that the retrieved documents actually corresponded to the patent-cited references they were linked with.

The SCImago Research Group has developed a procedure for linking the NPL references of the patents indexed in the PATSTAT database to the scientific papers indexed in the Scopus bibliographic database. This procedure has been implemented with reasonable results and assumable costs (Guerrero-Bote, Sánchez-Jiménez, & Moya-Anegón, 2019; Moya-Anegón et al., 2020). It can be summarized as follows:

Data pre-processing: Preparation of the data to facilitate and streamline the subsequent processes.

Pre-selection of candidate pairs: Some coincidences of the elements of NPL references in PATSTAT and publication records in Scopus are used to preselect candidates for a matching pair.

Automated evaluation of candidate pairs: The matching elements of each candidate pair are evaluated, and the pair is assigned a score. Thus, for each NPL reference, an ordered list is obtained of the Scopus references that could match it.

Human validation: For each NPL reference, the top-ranked Scopus reference is checked manually, ensuring that they really do match.

Since human validation plays a key role in the matching process, the precision was clearly going to be very high, but it still cannot be claimed to be 100%. In the spring 2019 version of PATSTAT (the one used for this study), patent applications from 2003 or later include 54 million NPL references, of which 20 million are to documents dated 2003 or later, and, of these, 7.4 million have been linked following the above procedure. Few of the remaining 12.6 million references are scientific papers included in Scopus: from a manual check of a sample stratified by year of 600 of these remaining references, we estimated that only 0.37 million of them are actually documents included in Scopus, which would represent a recall of about 95%.

Methods

In general terms, a patent family can be said to be a group of related patents. But there are different specific definitions of the concept (Martínez, 2011). The EPO (European Patent Office, 2019), for instance, uses two definitions:

Simple family, DOCDB family, or Espacenet patent family: All applications of the family have the same priorities.

Extended family or INPADOC family: All applications of the family are linked to the same root priority application. There may be applications in the same family that have no priority in common, but do have one in common with a third application which is the one that unites them.

In the present work, for convenience our focus is on the concept of a Simple family, which groups together all the applications corresponding to the same invention (European Patent Office, 2019).

As our goal was to develop indicators of the technological impact of recent scientific literature, we decided to take all patent applications into account, regardless of whether or not they had been granted. The reason was that if an applicant had taken the necessary steps to start the procedure then it was because the inventor and/or applicant considered the invention to be commercially exploitable, regardless of whether for legal (e.g. the application was late) or other reasons the patent would ultimately be granted. It should also be noted that many patents are not exploited at all or are exploited only very briefly, which makes it hard to determine their value.

Just as not all scientific papers are equally important, not all patent applications have the same weight. Although there have been many studies on indicators of patent value, such as those relating to patent citations or renewals, these indicators require time to manifest themselves, i.e. time to be cited or time to be renewed. On top of this, there is the time necessary for a patent application to be published. This is why we elected to weight patent applications according to the countries/regions in which protection was being sought.

In order to differentiate among patent families according to this criterion, we decided to assign a weight to each country/region. We explored three candidate indicators with which to generate this weight: the number of patent applications in the country/region, the country/region's GDP, and its investment in R&D. The first (the number of patent applications in a country/region) was discarded because of the existence of very small countries/regions with a small market but a disproportionate number of patents requesting protection, this being a consequence of regional offices and treaties such as the Eurasian Patent Organization (EAPO), the European Patent Office (EPO), and the European Patent Convention (EPC). The third (investment in R&D) was ruled out due to the difficulty of getting reliable annual data and because there is no direct relationship between the amount of R&D investment and patent applications. We therefore chose as the best option a weighting factor based on a country/region's GDP as an indicator of the commercial potential that was to be expected for that country/region. Specifically, a weight was defined for each country/region and for each particular year that was equal to its share of World GDP. The weight associated with a patent application is then determined by the country/region in which protection is requested in the filing year.

To weight citations from a patent family, an alternative approach would have been to use the absolute number of patents in the family or the number of designated countries/regions as the weighting factor. But such an approach would not take into account the state of economic development of the designated countries/regions. These criteria were therefore not used in the definition of the indicators that we finally explored.

While the NPL references may have different origins (applicant, examiner, opponent, etc.), no such differentiation was made in the present work. As will be outlined further below, both the number and the origin of NPL references are influenced by each national office's evaluation procedure, and this varies significantly from one country/region to another. Since patent families constitute the source or citing side of the citation process, duplicate references to scientific papers within each patent family's applications were merged into one. Also, to allow for differences among patents in the number of references to scientific papers, the aforementioned country/region weighting factor was divided by the total number of papers cited in each patent family.

For instance, for a patent family which makes r citations to scientific papers, and which designates countries/regions c1 and c2 which have shares of World GDP equal to GDP.c1 and GDP.c2, respectively, the weighting factors w1 and w2 for those countries/regions c1 and c2 are calculated as follows: w1=GDP.cir;w2=GDP.c2r.{w_1} = {{GDP.{c_i}} \over r};{w_2} = {{GDP.{c_2}} \over r}.

In the tables below, a weighting factor based solely on a country/region's share of World GDP will be denoted as a GDP-related weight, and a weighting factor resulting from dividing this share by the total number of a patent family's unique citations to papers as a fractional GDP-related weight.

In Table 1, patent families constitute the source or citing side of the citation process, and their citations to scientific papers are referred to as cites in the column headings. Articles published in the scientific literature constitute the target or cited side of the process, and are referred to as papers in the column headings. The countries/regions in the 1st column are the 28 designated nations of the patent applications with the greatest weighted number of citations to scientific papers (see the last column). These 28 countries/regions together account for 98% of the total weighted number of citations to scientific papers.

Patent citation based indicators by designated country/region.

Designated country/region# Patent FamiliesFamilies with cites to papers (%)# Pat Fam cites to papersAvg cites to papers per Pat FamWeighted avg GDP shareAvg cites to papers per Pat Fam with ≥1 citesPat Fam with filing year ≥2003 (%)Pat Fam cites to papers (%)Weighted cites to papers (%)
United States540085711.6535330760.650.23433.2922.4781.1545.91
China85719054.2414632740.170.10833.1235.6633.6112.50
Germany240877515.7921585160.900.05235.0210.0249.586.18
Japan50963854.6514102940.280.07938.6121.2032.395.88
United Kingdom203746818.0221366751.050.04135.478.4849.084.72
France188791119.1521050691.120.04035.667.8548.354.47
Italy183279619.2820857141.140.03135.947.6247.913.49
Spain176219220.0820867191.180.02135.927.3347.932.28
Netherlands175331820.0220785751.190.01336.037.2947.741.37
Turkey174279220.0520747341.190.01136.087.2547.661.22
Canada51316625.8911110762.170.02447.312.1325.520.96
Switzerland174530520.0520760161.190.00936.067.2647.690.95
Sweden174821520.0120757241.190.00836.077.2747.680.82
Belgium173214820.1820750831.200.00736.087.2147.660.78
South Korea23480835.968154120.350.01841.869.7718.730.78
Poland170743020.2020593671.210.00736.217.1047.300.74
Austria174299220.0720758911.190.00636.077.2547.680.65
Norway126490422.1017722021.400.00638.365.2640.710.55
Denmark174015220.0920762181.190.00536.087.2447.690.52
Brazil29679020.854782601.610.02947.001.2310.990.52
Australia44627821.888577251.920.01849.691.8619.700.52
Greece173113020.1920747961.200.00436.087.2047.660.45
Finland174124720.1020758831.190.00436.077.2447.680.41
Ireland173018720.2020748651.200.00436.087.2047.660.41
Portugal172849520.2320761241.200.00336.087.1947.690.37
Czech Republic173403020.1820759231.200.00336.067.2147.680.32
Russian Federation5343687.922753690.520.02543.572.226.330.31
Romania173375320.1520743211.200.00336.097.2147.650.27

Legend to Table 1:

Designated country/region: The country/region designated in a patent application.

# Patent Families: The number of PATSTAT patent families applying for protection in the country/region.

Families with cites to papers (%): Percentage of patent families that include citations to scientific papers indexed in Scopus.

# Pat Fam cites to papers: The number of citations in patent families to scientific papers indexed in Scopus.

Avg cites to papers per Pat Fam: The average number of citations to scientific papers indexed in Scopus per patent family (the ratio between the previous two numbers).

Weighted avg GDP share: The country/region's time-averaged share of World GDP calculated by weighting the share in each year by its number of patent applications in that year.

Avg cites to papers per Pat Fam with ≥1 cites: The average number of citations to scientific papers indexed in Scopus in patent families that make at least one such citation.

Pat Fam with filing year ≥2003 (%): Percentage of patent families with filing year 2003 or later relative to the total number of patent families.

Pat Fam cites to papers (%): Percentage of patent family citations to scientific papers indexed in Scopus relative to the total number of patent family citations to papers.

Weighted cites to papers (%): Weighted citations to papers (expressed as a percentage of the total over all countries/regions).

One observes in Table 1 that 5,400,857 patent families have the United States as a designated country, although of course they may designate other countries/regions as well. These families contain 3,533,076 citations to scientific papers indexed in Scopus, so that the average number of citations to papers per patent family is 0.65. Only a small percentage of patent families contain citations to papers, so that when only these families (those with at least one citation to papers) are considered then the average number of citations to papers per patent family rises to 33.29.

As mentioned above, for each patent family designating a given country/region, a weighting factor is calculated which is defined as the ratio between the designated country/region's share of World GDP and the patent family's total number of citations to papers. For example, calculating this ratio for all families designating the United States and having at least one citation to papers for each year considered, and computing the average over those years, one obtains a value of 0.234. This number can be interpreted as the “value” of a citation in a patent family designating the United States to an arbitrary scientific paper. It therefore relates to the citing side of the citation process, not to the cited side. It will be used below as a type of source normalization factor in calculating a scientific paper's technological impact.

The “percentage weighted cites to papers” in the last column of Table 1 is calculated in the following manner. First, for each designation country/region the weighted number of citations to papers is calculated by multiplying the number of citations in patent families designating that country/region by its normalization factor. Next, these weighted numbers of citations are summed over all designation countries/regions. Finally, for each country/region, the share of weighted citations is computed relative to the sum over all countries/regions, expressed as a percentage.

Although it is unsurprising that the United States is the country with the largest share of weighted citations, it has fewer patent families than China and only slightly more than Japan. The main difference stems from the absolute number of patent citations to scientific papers, as can be seen in the 4th column. Specifically, in China and in Japan there must be many patents that do not cite any scientific paper indexed in Scopus.

In the 8th and 9th columns, one can clearly see the differences between a country/region's percentage of patent families and its percentage of patent family citations. Keep in mind that there are many patent families that apply for protection in multiple countries/regions, which explains why these percentages sum to more than 100%.

One also observes that there are differences between countries/regions in the average number of patent citations to papers. These reflect the differences among the patent authorities regarding their requirements and review processes. It can be seen that while the United States accounts for almost 45% of the weighted cites to papers, only three other countries/regions exceed 5%.

Table 2 presents the data listed by the four “citable” document types: Article, Conference Paper, Review, and Short Survey. It shows that the most cited document type is Article followed by Conference Paper. However, although the Article is the most frequently occurring document type, if one considers the average number of citing patent families per paper (3rd column), the Short Survey ranks first, followed by the Conference Paper, the Review, and the Article. The 4th column lists the average number of citations to scientific papers indexed in Scopus that are included in patents that cite a particular document type. The Conference Paper receives on average less than half the number of citations received by the other types. The 5th column gives the average number of citations weighted with the share of global GDP of designated countries/regions in patent families that cite a particular type of document, and the 6th column the average number of citations weighted with the fractional weight, as described above, defined as the ratio between a country/region's share of global GDP and the total number of citations to papers in a citing patent family.

Patent citation according to document type of the cited paper.

Document type# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
Article6708310.00838.90.0200.0023
Conference Paper3405770.01817.40.0250.0048
Review1771720.01247.10.0180.0019
Short Survey245800.02048.10.0180.0013

Legend to Table 2:

# Patent families: Number of citing patent families.

Avg cites to papers: Average number of patent families citing any given document of that type.

Avg refs in patents: Average number of citations to papers in patent families which cite that document type.

Avg cites to papers (GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP.

Avg cites to papers (fractional GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP divided by the total number of a patent family's unique citations to papers.

It is striking that Conference Papers have the greatest average fractional weight. This can be explained by the fact that in fields like computer science conference papers have greater relevance in combination with the observation (obtained from a secondary analysis not presented in the present communication) that the United States is the country with the greatest percentage of citations to conference papers as well as the greatest GDP-related weight of its patent families.

Table 3 is similar to Table 2, but relates to the cited papers’ publication year rather than document type. It shows declines in the number of patent families, the average number of citations weighted with both the GDP share and the fractional weight. The more recent the paper, the less time it has had to be cited, as is true also for scientific paper-to-paper citations. The average number of citations to papers in the citing patents grew until 2013. This was to be expected since old patents have a smaller time frame from which to pick papers to cite (only post-2003 papers were considered) than that of more recently published patents.

Patent citation according to publication year of the cited paper.

Year# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
20032303080.01931.490.02180.00400
20042329050.01933.770.02120.00350
20052296440.01935.010.02080.00318
20062148220.01935.430.02030.00292
20072057470.01937.530.02020.00278
20081926820.01939.030.02010.00261
20091771980.01939.740.01950.00255
20101604830.01940.160.01930.00249
20111404610.01941.070.01910.00244
20121167530.01842.900.01890.00233
2013879780.01646.070.01880.00207
2014615860.01442.730.01860.00178
2015382250.01139.680.01850.00134
2016180820.00642.640.01890.00064
201760460.00242.450.02020.00022
20188060.00038.900.01910.00003

Legend to Table 3:

# Patent families: Number of citing patent families.

Avg cites to papers: Average number of patent families citing a document type.

Avg refs in patents: Average number of citations to papers in patent families citing a document type.

Avg cites to papers (GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP.

Avg cites to papers (fractional GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP divided by the total number of a patent family's unique citations to papers.

Table 4 is similar to Table 3, but gives results by Scientific Area. As expected, there are large differences between areas.

Patent citation according to Scientific Area of the cited paper.

Scientific Area# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
Computer Science2516490.01916.100.0250.00525
Engineering3947490.01821.550.0240.00466
Materials Science2286700.01823.880.0240.00428
Physics and Astronomy2048020.01724.030.0250.00401
Energy560340.02015.610.0220.00395
Mathematics1116520.02118.100.0240.00386
Chemistry2411280.01527.280.0200.00350
Chemical Engineering1544590.01733.270.0200.00309
Decision Sciences118590.02113.790.0280.00280
Health Professions210040.01729.710.0210.00247
Environmental Science510470.01524.880.0200.00230
Pharmacology, Toxicology and Pharmaceutics1200680.01140.290.0180.00227
Agricultural and Biological Sciences789730.01234.360.0180.00193
Dentistry40800.01128.900.0190.00192
Biochemistry, Genetics and Molecular Biology2714920.00745.620.0180.00187
Earth and Planetary Sciences193570.00915.670.0220.00172
Medicine2707030.00646.070.0180.00164
Multidisciplinary671840.01755.330.0190.00161
Immunology and Microbiology944140.00946.120.0180.00160
Business, Management and Accounting102290.01011.890.0300.00154
Veterinary106190.01232.960.0180.00153
Neuroscience438460.01155.510.0190.00141
Nursing136880.01154.480.0180.00132
Arts and Humanities378960.01550.430.0210.00128
Social Sciences228000.00719.650.0250.00100
Economics, Econometrics and Finance20440.00411.760.0340.00062
Psychology60260.00663.150.0210.00059

Legend to Table 4:

# Patent families: Number of citing patent families.

Avg cites to papers: Average number of patent families citing a document type.

Avg refs in patents: Average number of citations to papers in patent families citing a document type.

Avg cites to papers (GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP.

Avg cites to papers (fractional GDP-based weighting): Average number of citations weighted by a factor defined as a country/region's share of global GDP divided by the total number of a patent family's unique citations to papers.

A general conclusion to be drawn from Tables 1–4 is that a large part of the scientific papers receive no citations from patents, and that patent citations are very unevenly distributed among the scientific areas, document types, and cited publication years.

Proposed indicator for countries/regions and other aggregates

In developing a proposal for an indicator, we were inspired by the item-oriented, field-normalized, citation-score average indicator introduced by Lundberg (2007) which formed the base for the CWTS’ mean normalized citation score (Waltman et al., 2011) and SCImago's normalized impact (SCImago Research Group, 2010). In this case however, we did not assume a priori that scientific contributions in all subject fields have the same technological value. It is clearly plausible that a contribution in Engineering or Pharmacology would have a greater technological impact than a contribution in the Humanities. We therefore decided not to implement any subject field normalization. Similar reasoning led us to also reject normalization according to document type.

Hence, we defined the Technological Impact of an aggregate as TI=1ni=1naiei;ai=j=1pwij{\rm{TI}} = {1 \over n}\sum\limits_{i = 1}^n {{{{a_i}} \over {{e_i}}};{a_i} = \sum\limits_{j = 1}^p {{w_{ij}}} } where n is the number of (cited or not) papers published by the aggregate (e.g. an institution, a journal, a country/region), ai is the sum of the weights of the patent citations received by paper i, ei is the expected value of the sum of weights for papers of the same age, p is the number of patent families citing the i-th paper (p depends on i), wij is the weight of the citation from patent family j received by the i-th paper as defined in Section 3: wij=GDP.cr{w_{ij}} = {{GDP.c} \over r} in which GDP represents the World GDP, c a designated country/region's share of the World GDP, and r the number of citations in a patent family to scientific papers indexed in Scopus. The indicator TI is size-independent and, as a measure, corresponds to the average citation obtained by the aggregate's papers.

As an analogue of the total normalized citation score indicator known as “Brute Force”, we define Technological Force as: TFO=nTI=i=1naiei.{\rm{TFO}} = n \cdot {\rm{TI}} = {{\sum\limits_{i = 1}^n {{a_i}} } \over {{e_i}}}.

This indicator is size-dependent, and is a measure of the total technological contribution of the aggregate.

Results

In analysing the behaviour of the indicators, only results for the main countries/regions will be presented. We shall begin by focusing on TFO as a size-dependent indicator, and compare it with other size-dependent measures of a country/region's performance such as the number of patent families that have applied for protection in the country/region, its scientific publication output, a “brute force” indicator defined as the product of the publication output and the field-normalized citation impact, its scientific excellence output (Top 10% and Top 1%), and total citations from patent families to its scientific output. To make these size-dependent indicators more readily comparable, they will be expressed as percentages with respect to the global total. These percentages may sum to more than 100% because there are families that apply for protection in more than one country/region, and papers may be published in collaboration among several countries/regions.

Table 5 presents the scores of the 28 countries/regions with the highest value of TFO. In all indicators, the United States and China rank top, although not in the same proportions. In the 2nd column, which gives the percentage of families applying for protection in a country/region, one notes that China ranks higher than the United States. Also visible is the effect of the EPC. While in the past a designation fee was paid that depended on the designated countries/regions, since 2007 all European patent applications pay a single fee and designate all countries/regions. This causes many patents to seek protection in a large number of European Union countries, with the result being that around 7% of all patent families apply for protection in almost all countries of the EU.

Size-dependent indicators of the 28 top-ranked %TFO countries/regions.

Country/Region%Families%Output%BF%Exc10%Exc1%Fam.Cit.%TFO
United States22.4724.2536.3737.7646.4543.7436.88
China35.6614.0511.4813.0913.586.6814.32
Germany10.026.138.328.7110.057.938.31
Japan21.205.214.804.594.107.647.83
United Kingdom8.487.0510.8511.2314.037.196.50
France7.854.305.575.866.564.594.74
South Korea9.772.502.532.712.653.514.56
Canada2.133.665.505.767.164.254.40
Italy7.623.655.015.265.823.273.29
India0.233.803.062.892.521.532.74
Netherlands7.292.023.553.905.222.642.56
Spain7.332.973.713.984.412.192.49
Switzerland7.261.502.702.914.112.482.48
Australia1.862.974.524.836.052.222.36
Chinese Taiwan2.741.441.431.581.341.592.19
Sweden7.271.332.162.312.931.701.65
Belgium7.211.121.811.942.581.471.48
Singapore0.310.661.111.251.790.971.33
Denmark7.240.831.491.592.181.031.12
Austria7.250.821.231.271.631.051.10
Israel0.320.751.101.151.411.181.03
Finland7.240.691.061.101.330.831.03
Chinese Hong Kong0.300.691.131.311.710.791.00
Brazil1.232.141.841.661.550.750.92
Russian Federation2.222.121.361.061.040.600.78
Poland7.101.411.231.071.150.590.76
Norway5.260.671.081.081.370.510.72
Iran0.001.281.161.281.170.330.65

Legend to Table 5:

All the percentages are relative to the global total.

%Families: Percentage of PATSTAT patent families with filing year 2003 or later applying for protection in the country/region.

%Output: Percentage of scientific papers indexed in Scopus.

%BF: Brute Force percentage, i.e., %Output × Normalized Impact (NI).

%Exc10: Percentage of scientific papers of excellence (Top 10%).

%Exc1: Percentage of scientific papers of excellence (Top 1%).

%Fam.Cit.: Percentage of patent family citations received.

%TFO: Percentage Technological Force.

One observes in the table that China has a smaller percentage of patent family citations than would have been expected given its total publication output. This is because patents that apply for protection in China are characterized by making few citations to papers, and those that cite papers from China itself even fewer. Nonetheless, these patents have an above-average weight precisely due to this low number of references and because China's GDP is the world's second largest, meaning that its cited references also contribute more, leading to a TFO within the expected range.

Figure 1 shows the same data as presented in Table 5 for the 12 countries/regions with the highest values of Technological Force (We plot just 12 countries/regions so that the profiles can be clearly seen.). The figure shows a large number of families applying for protection in China, Japan, and South Korea. Also, the EPC effect can be seen, with the other EU countries following in the wake of Germany and the United Kingdom in the number of patent families applying for protection. It is notable that India has a different profile, with relatively few patent families applying for protection in its territory.

Figure 1

Size-dependent indicators of the 12 countries with greatest Technical Force. Percentages relative to the global total.

% Families: Percentage of PATSTAT patent families with filing year 2003 or later applying for protection in the country/region.

% Output: Percentage of scientific papers indexed in Scopus.

% BF: Brute Force percentage, i.e., %Output × Normalized Impact (NI).

% Exc10: Percentage of scientific papers of excellence (Top 10%).

% Exc1: Percentage of scientific papers of excellence (Top 1%).

% Fam. Cit: Percentage of patent family citations received.

% TFO: Percentage of Technological Force.

Table 5 reveals Australia's low position in terms of the percentage of patent families citing its papers, and the EPC effect in Sweden and Belgium in this sense. Above all, one notes the high position of South Korea in the table (7th), while it ranks just 14th in terms of scientific output.

Table 6 gives the Pearson correlation coefficients between the size-dependent indicators of the 40 countries/regions with the greatest scientific outputs. The indicator that correlates least well with the others is Families, that corresponding to patent families applying for protection. There is a strong correlation between the indicators that combine quantity and quality of scientific production (BF, Exc10, and Exc1). Technological Force has a stronger correlation with all the other indicators than the direct values of patent family citations.

Pearson correlation matrix between the size-dependent indicators for the 40 countries/regions with the greatest scientific output.

%Families%Output%BF%Exc10%Exc1%Fam.Cit.%TFO
%Families10.720.600.610.580.550.68
%Output0.7210.970.970.950.920.987
%BF0.600.9710.990.990.980.98
%Exc100.610.970.9910.990.970.98
%Exc10.580.950.990.9910.970.97
%Fam.Cit.0.540.920.980.970.9710.97
%TFO0.680.980.980.980.970.971

The analyses presented in Tables 7 and 8 are analogous to those of Tables 5 and 6, but based on size-independent rather than size-dependent indicators. The scores in these tables are relative to the global average, not the actual values of the indicators. The same set of 28 countries/regions are considered, i.e., those with the largest Technological Force (TFO), and they listed in Table 7 in the same order as in Table 5. But the actual ranking based on Technological Impact (TI) differs strongly from that based on TFO. Indeed, the top 10 countries/regions ranked by TI would be Singapore (1.97), South Korea (1.67), Switzerland (1.62), United States (1.55), Finland (1.44), Chinese Hong Kong (1.41), Chinese Taiwan (1.40), Japan (1.40) Israel (1.35), and Denmark (1,33).

Size-independent indicators of the 28 top-ranked %TFO countries/regions.

Country/RegionFam.Rel.%Q1NI%Exc10%Exc1Avg.Fam.Cit.TI
United States2.991.381.501.682.241.981.55
China2.970.830.821.001.130.460.92
Germany7.141.241.361.531.921.351.32
Japan5.231.090.920.950.921.471.40
United Kingdom6.001.371.541.722.331.170.98
France9.691.261.301.471.791.111.06
South Korea6.441.131.011.171.241.381.67
Canada4.171.391.501.692.291.251.20
Italy11.141.251.371.551.870.950.88
India0.180.700.810.820.780.410.69
Netherlands20.031.551.762.083.041.401.26
Spain13.701.241.251.441.740.770.81
Switzerland26.891.491.812.093.211.751.62
Australia3.781.381.521.752.390.810.80
Chinese Taiwan5.221.200.991.181.081.091.40
Sweden30.301.521.631.872.581.341.20
Belgium36.041.421.621.872.711.391.30
Singapore4.021.341.692.063.201.541.97
Denmark48.571.541.802.073.091.321.33
Austria48.831.271.501.662.321.351.30
Israel4.361.461.471.662.201.671.35
Finland58.691.381.541.732.271.251.44
Chinese Hong Kong3.531.391.642.062.921.201.41
Brazil3.330.880.860.840.850.360.40
Russian Fed.2.300.490.640.540.580.280.33
Poland28.160.790.870.820.950.420.50
Norway47.681.371.601.732.370.811.07
Iran0.000.740.911.081.070.250.47

Legend to Table 7.

All indicators are relative to the global average, so that a value of unity would correspond to that global average.

Fam.Rel.: Ratio between the number of patent families applying for protection in the country/region and its number of scientific papers.

%Q1: Percentage of scientific papers in the first quartile.

NI: Normalized Impact.

%Exc10: % Excellence 10 (Top 10%).

%Exc1: % Excellence 1 (Top 1%).

Avg.Fam.Cit.: Average number of patent family citations per paper.

TI: Technological Impact.

Pearson correlation matrix between size-independent indicators for the 40 countries/regions with the greatest scientific output.

Fam.Rel.%Q1NI%Exc10%Exc1Avg.Fam.Cit.TI
Fam.Rel.10.200.250.180.210.08−0.00
%Q10.2010.920.920.860.830.74
NI0.250.9210.990.980.740.69
%Exc100.180.920.9910.980.750.73
%Exc10.210.860.980.9810.700.69
Avg.Fam.Cit.0.080.830.740.750.7010.90
TI−0.000.740.690.730.690.901

Figure 2 is a scatter plot of Technological Impact (TI) vs the average number of patent family citations received (Avg.Fam.Cit.) for the 28 countries/regions with the greatest values of Technical Force (TFO). While TI is an advanced, normalized indicator, Avg.Fam.Cit. is a simpler, raw indicator. The plot clearly shows that Asian countries/regions in particular have relatively higher positions on the TI scale than on the Avg.Fam.Cit. scale. This means that TI indeed does make a difference in revealing features that remain hidden in analyses of straight patent citation counts.

Figure 2

Technological Impact (TI) vs average number of patent family citations received per paper (Avg. Fam.Cit.) of the 28 countries/regions with the highest values of Technical Force.

Figure 3 is the size-independent analogue of Figure 1. The profiles it shows are more homogeneous than those shown in Figure 1 with there only being more variations in the average number of patent family citations.

Figure 3

Size-independent indicators of the 12 countries with greatest Technical Force.

All indicators are relative to the global average.

% Q1: Percentage of scientific papers in Q1.

NI: Normalized Impact.

% Exc10: Percentage of scientific papers of excellence (Top 10%).

% Exc1: Percentage of scientific papers of excellence (Top 1%).

Fam. Cit. Avg.: Average number of patent family citations per paper.

TI: Technological Impact.

The correlation matrix in Table 8 shows that Fam.Rel. (the ratio between a country/region's global share of patent families applying for protection and its global share of scientific papers) is the least correlated with the other indicators. The quality indicators of scientific production (% Q1, NI, % Exc10, and % Exc1) have greater correlations with each other, and Technological Impact logically correlates more strongly with the average number of patent citations. Technological Impact correlates a little more strongly with % Q1 and % Exc10 than with the other two indicators of scientific quality. In any case, the correlations of TI are weaker (around 0.7) than those of TFO with the rest of the size-dependent indicators (around 0.9).

Figure 4 is a scatter plot of the countries/regions according to technological impact and scientific impact. One observes that Singapore has a high scientific impact, but above all stands out for its technological impact. Australia and the United Kingdom have significant scientific impact, but technological impact below the global average. Germany and Canada have lower scientific impact but greater technological impact. Japan, Chinese Taiwan, and South Korea stand out for their technological impact but not for their scientific impact.

Figure 4

Scatter plot of Technological Impact vs Scientific Impact of the 28 countries/regions with the highest values of TFO. The circumferences correspond to: Technological Force (the outer thin circumference), Excellence 10 (thick circumference), the number of patent family citations (the inner thin circumference).

The relative sizes of the circumferences also mark different profiles. While in Japan the thick circumference (Exc10) is the smallest, smaller than the two thin ones (TFO and patent family citations), in China the smallest is that of patent family citations. And in Australia and the United Kingdom, the largest circumference (TFO) is proportionally closer to the other two than in other countries/regions.

Figure 5 shows the temporal evolution of Technological Impact, Normalized Impact, and the average number of patent family citations for the two countries with the greatest scientific output and the countries/regions of the present authors. While the average citation rate from patent families has been in steady decline, TI has been stable except in the last two years. This loss of stability is due to the time it takes for the authorities to publish patent applications.

Figure 5

Annual evolution of three indicators—Normalized Impact (NI), average number of patent family citations (Fam.Cit. Avg.), and Technological Impact (TI)—in four countries.

Conclusions

The proposed indicators, Technological Impact (TI) and Technological Force (TFO), are designed to complement each other—the first being size-independent, and the second size-dependent. Their construction takes into account the countries/regions in which patent protection was sought, a patent family's propensity to cite scientific papers, merging and deduplication of the reference lists of applications corresponding to the same invention, and the time separating the publication date of a scientific paper and its subsequent citation in a patent.

Similar to the Normalized Scientific Impact indicator which normalizes the number of citations in scientific papers-to other papers with the average citation rate of articles in the same subject field and with the same publication year and document type, these advanced indicators of technological impact, being normalized with the scores corresponding to papers of the same age, are capable of partially eliminating the inevitable decline in patent citations to papers published in more recent years. It has to be noted, however, that the indicator TI was found to lose stability in the last two years due to the time that it takes for the various authorities to publish patent applications.

The indicators explored in the present communication show strong linear correlations with the indicators of scientific production, but are also able to reveal features that remain hidden in analyses of straight patent citation counts. An example was seen in showing the uniqueness of such technologically advanced countries as Singapore, South Korea, or Japan whose R&D systems are better integrated with innovation.

The proposed indicators are useful tools with which to characterize the technological orientation of research institutions.

Figure 1

Size-dependent indicators of the 12 countries with greatest Technical Force. Percentages relative to the global total.% Families: Percentage of PATSTAT patent families with filing year 2003 or later applying for protection in the country/region.% Output: Percentage of scientific papers indexed in Scopus.% BF: Brute Force percentage, i.e., %Output × Normalized Impact (NI).% Exc10: Percentage of scientific papers of excellence (Top 10%).% Exc1: Percentage of scientific papers of excellence (Top 1%).% Fam. Cit: Percentage of patent family citations received.% TFO: Percentage of Technological Force.
Size-dependent indicators of the 12 countries with greatest Technical Force. Percentages relative to the global total.% Families: Percentage of PATSTAT patent families with filing year 2003 or later applying for protection in the country/region.% Output: Percentage of scientific papers indexed in Scopus.% BF: Brute Force percentage, i.e., %Output × Normalized Impact (NI).% Exc10: Percentage of scientific papers of excellence (Top 10%).% Exc1: Percentage of scientific papers of excellence (Top 1%).% Fam. Cit: Percentage of patent family citations received.% TFO: Percentage of Technological Force.

Figure 2

Technological Impact (TI) vs average number of patent family citations received per paper (Avg. Fam.Cit.) of the 28 countries/regions with the highest values of Technical Force.
Technological Impact (TI) vs average number of patent family citations received per paper (Avg. Fam.Cit.) of the 28 countries/regions with the highest values of Technical Force.

Figure 3

Size-independent indicators of the 12 countries with greatest Technical Force.All indicators are relative to the global average.
% Q1: Percentage of scientific papers in Q1.NI: Normalized Impact.% Exc10: Percentage of scientific papers of excellence (Top 10%).% Exc1: Percentage of scientific papers of excellence (Top 1%).Fam. Cit. Avg.: Average number of patent family citations per paper.TI: Technological Impact.
Size-independent indicators of the 12 countries with greatest Technical Force.All indicators are relative to the global average. % Q1: Percentage of scientific papers in Q1.NI: Normalized Impact.% Exc10: Percentage of scientific papers of excellence (Top 10%).% Exc1: Percentage of scientific papers of excellence (Top 1%).Fam. Cit. Avg.: Average number of patent family citations per paper.TI: Technological Impact.

Figure 4

Scatter plot of Technological Impact vs Scientific Impact of the 28 countries/regions with the highest values of TFO. The circumferences correspond to: Technological Force (the outer thin circumference), Excellence 10 (thick circumference), the number of patent family citations (the inner thin circumference).
Scatter plot of Technological Impact vs Scientific Impact of the 28 countries/regions with the highest values of TFO. The circumferences correspond to: Technological Force (the outer thin circumference), Excellence 10 (thick circumference), the number of patent family citations (the inner thin circumference).

Figure 5

Annual evolution of three indicators—Normalized Impact (NI), average number of patent family citations (Fam.Cit. Avg.), and Technological Impact (TI)—in four countries.
Annual evolution of three indicators—Normalized Impact (NI), average number of patent family citations (Fam.Cit. Avg.), and Technological Impact (TI)—in four countries.

Patent citation according to publication year of the cited paper.

Year# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
20032303080.01931.490.02180.00400
20042329050.01933.770.02120.00350
20052296440.01935.010.02080.00318
20062148220.01935.430.02030.00292
20072057470.01937.530.02020.00278
20081926820.01939.030.02010.00261
20091771980.01939.740.01950.00255
20101604830.01940.160.01930.00249
20111404610.01941.070.01910.00244
20121167530.01842.900.01890.00233
2013879780.01646.070.01880.00207
2014615860.01442.730.01860.00178
2015382250.01139.680.01850.00134
2016180820.00642.640.01890.00064
201760460.00242.450.02020.00022
20188060.00038.900.01910.00003

Patent citation according to document type of the cited paper.

Document type# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
Article6708310.00838.90.0200.0023
Conference Paper3405770.01817.40.0250.0048
Review1771720.01247.10.0180.0019
Short Survey245800.02048.10.0180.0013

Pearson correlation matrix between size-independent indicators for the 40 countries/regions with the greatest scientific output.

Fam.Rel.%Q1NI%Exc10%Exc1Avg.Fam.Cit.TI
Fam.Rel.10.200.250.180.210.08−0.00
%Q10.2010.920.920.860.830.74
NI0.250.9210.990.980.740.69
%Exc100.180.920.9910.980.750.73
%Exc10.210.860.980.9810.700.69
Avg.Fam.Cit.0.080.830.740.750.7010.90
TI−0.000.740.690.730.690.901

Pearson correlation matrix between the size-dependent indicators for the 40 countries/regions with the greatest scientific output.

%Families%Output%BF%Exc10%Exc1%Fam.Cit.%TFO
%Families10.720.600.610.580.550.68
%Output0.7210.970.970.950.920.987
%BF0.600.9710.990.990.980.98
%Exc100.610.970.9910.990.970.98
%Exc10.580.950.990.9910.970.97
%Fam.Cit.0.540.920.980.970.9710.97
%TFO0.680.980.980.980.970.971

Patent citation based indicators by designated country/region.

Designated country/region# Patent FamiliesFamilies with cites to papers (%)# Pat Fam cites to papersAvg cites to papers per Pat FamWeighted avg GDP shareAvg cites to papers per Pat Fam with ≥1 citesPat Fam with filing year ≥2003 (%)Pat Fam cites to papers (%)Weighted cites to papers (%)
United States540085711.6535330760.650.23433.2922.4781.1545.91
China85719054.2414632740.170.10833.1235.6633.6112.50
Germany240877515.7921585160.900.05235.0210.0249.586.18
Japan50963854.6514102940.280.07938.6121.2032.395.88
United Kingdom203746818.0221366751.050.04135.478.4849.084.72
France188791119.1521050691.120.04035.667.8548.354.47
Italy183279619.2820857141.140.03135.947.6247.913.49
Spain176219220.0820867191.180.02135.927.3347.932.28
Netherlands175331820.0220785751.190.01336.037.2947.741.37
Turkey174279220.0520747341.190.01136.087.2547.661.22
Canada51316625.8911110762.170.02447.312.1325.520.96
Switzerland174530520.0520760161.190.00936.067.2647.690.95
Sweden174821520.0120757241.190.00836.077.2747.680.82
Belgium173214820.1820750831.200.00736.087.2147.660.78
South Korea23480835.968154120.350.01841.869.7718.730.78
Poland170743020.2020593671.210.00736.217.1047.300.74
Austria174299220.0720758911.190.00636.077.2547.680.65
Norway126490422.1017722021.400.00638.365.2640.710.55
Denmark174015220.0920762181.190.00536.087.2447.690.52
Brazil29679020.854782601.610.02947.001.2310.990.52
Australia44627821.888577251.920.01849.691.8619.700.52
Greece173113020.1920747961.200.00436.087.2047.660.45
Finland174124720.1020758831.190.00436.077.2447.680.41
Ireland173018720.2020748651.200.00436.087.2047.660.41
Portugal172849520.2320761241.200.00336.087.1947.690.37
Czech Republic173403020.1820759231.200.00336.067.2147.680.32
Russian Federation5343687.922753690.520.02543.572.226.330.31
Romania173375320.1520743211.200.00336.097.2147.650.27

Patent citation according to Scientific Area of the cited paper.

Scientific Area# Patent familiesAvg cites to papersAvg refs in patentsAvg cites to papers (GDP-based weighting)Avg cites to papers (fractional GDP-based weighting)
Computer Science2516490.01916.100.0250.00525
Engineering3947490.01821.550.0240.00466
Materials Science2286700.01823.880.0240.00428
Physics and Astronomy2048020.01724.030.0250.00401
Energy560340.02015.610.0220.00395
Mathematics1116520.02118.100.0240.00386
Chemistry2411280.01527.280.0200.00350
Chemical Engineering1544590.01733.270.0200.00309
Decision Sciences118590.02113.790.0280.00280
Health Professions210040.01729.710.0210.00247
Environmental Science510470.01524.880.0200.00230
Pharmacology, Toxicology and Pharmaceutics1200680.01140.290.0180.00227
Agricultural and Biological Sciences789730.01234.360.0180.00193
Dentistry40800.01128.900.0190.00192
Biochemistry, Genetics and Molecular Biology2714920.00745.620.0180.00187
Earth and Planetary Sciences193570.00915.670.0220.00172
Medicine2707030.00646.070.0180.00164
Multidisciplinary671840.01755.330.0190.00161
Immunology and Microbiology944140.00946.120.0180.00160
Business, Management and Accounting102290.01011.890.0300.00154
Veterinary106190.01232.960.0180.00153
Neuroscience438460.01155.510.0190.00141
Nursing136880.01154.480.0180.00132
Arts and Humanities378960.01550.430.0210.00128
Social Sciences228000.00719.650.0250.00100
Economics, Econometrics and Finance20440.00411.760.0340.00062
Psychology60260.00663.150.0210.00059

Size-dependent indicators of the 28 top-ranked %TFO countries/regions.

Country/Region%Families%Output%BF%Exc10%Exc1%Fam.Cit.%TFO
United States22.4724.2536.3737.7646.4543.7436.88
China35.6614.0511.4813.0913.586.6814.32
Germany10.026.138.328.7110.057.938.31
Japan21.205.214.804.594.107.647.83
United Kingdom8.487.0510.8511.2314.037.196.50
France7.854.305.575.866.564.594.74
South Korea9.772.502.532.712.653.514.56
Canada2.133.665.505.767.164.254.40
Italy7.623.655.015.265.823.273.29
India0.233.803.062.892.521.532.74
Netherlands7.292.023.553.905.222.642.56
Spain7.332.973.713.984.412.192.49
Switzerland7.261.502.702.914.112.482.48
Australia1.862.974.524.836.052.222.36
Chinese Taiwan2.741.441.431.581.341.592.19
Sweden7.271.332.162.312.931.701.65
Belgium7.211.121.811.942.581.471.48
Singapore0.310.661.111.251.790.971.33
Denmark7.240.831.491.592.181.031.12
Austria7.250.821.231.271.631.051.10
Israel0.320.751.101.151.411.181.03
Finland7.240.691.061.101.330.831.03
Chinese Hong Kong0.300.691.131.311.710.791.00
Brazil1.232.141.841.661.550.750.92
Russian Federation2.222.121.361.061.040.600.78
Poland7.101.411.231.071.150.590.76
Norway5.260.671.081.081.370.510.72
Iran0.001.281.161.281.170.330.65

Size-independent indicators of the 28 top-ranked %TFO countries/regions.

Country/RegionFam.Rel.%Q1NI%Exc10%Exc1Avg.Fam.Cit.TI
United States2.991.381.501.682.241.981.55
China2.970.830.821.001.130.460.92
Germany7.141.241.361.531.921.351.32
Japan5.231.090.920.950.921.471.40
United Kingdom6.001.371.541.722.331.170.98
France9.691.261.301.471.791.111.06
South Korea6.441.131.011.171.241.381.67
Canada4.171.391.501.692.291.251.20
Italy11.141.251.371.551.870.950.88
India0.180.700.810.820.780.410.69
Netherlands20.031.551.762.083.041.401.26
Spain13.701.241.251.441.740.770.81
Switzerland26.891.491.812.093.211.751.62
Australia3.781.381.521.752.390.810.80
Chinese Taiwan5.221.200.991.181.081.091.40
Sweden30.301.521.631.872.581.341.20
Belgium36.041.421.621.872.711.391.30
Singapore4.021.341.692.063.201.541.97
Denmark48.571.541.802.073.091.321.33
Austria48.831.271.501.662.321.351.30
Israel4.361.461.471.662.201.671.35
Finland58.691.381.541.732.271.251.44
Chinese Hong Kong3.531.391.642.062.921.201.41
Brazil3.330.880.860.840.850.360.40
Russian Fed.2.300.490.640.540.580.280.33
Poland28.160.790.870.820.950.420.50
Norway47.681.371.601.732.370.811.07
Iran0.000.740.911.081.070.250.47

Ahmadpoor, M., & Jones, B.F. (2017). The dual frontier: Patented inventions and prior scientific advance. Science, 357(6351), 583–587.AhmadpoorM.JonesB.F.2017The dual frontier: Patented inventions and prior scientific advanceScience3576351583587Search in Google Scholar

Archambault, E., Campbell, D., Gingras, Y., & Larivière, V. (2009). “Comparing bibliometric statistics obtained from the Web of Science and Scopus”. Journal of the American Society for Information Science and Technology, 60 (7), 1320–1326. https://doi.org/10.1002/asi.21062ArchambaultE.CampbellD.GingrasY.LarivièreV.2009“Comparing bibliometric statistics obtained from the Web of Science and Scopus”Journal of the American Society for Information Science and Technology60713201326https://doi.org/10.1002/asi.21062Search in Google Scholar

Bryan, K.A., Ozcan, Y., & Sampat, B. (2020). In-text patent citations: A user's guide. Research Policy, 49(4), 103946.BryanK.A.OzcanY.SampatB.2020In-text patent citations: A user's guideResearch Policy494103946Search in Google Scholar

Callaert, J., Pellens, M., & Van Looy, B. (2014). Sources of inspiration? Making sense of scientific references in patents. Scientometrics, 98(3), 1617–1629.CallaertJ.PellensM.Van LooyB.2014Sources of inspiration? Making sense of scientific references in patentsScientometrics98316171629Search in Google Scholar

Coffano, M., & Tarasconi, G. (2014). Crios—Patstat database: Sources, contents and access rules. Center for Research on Innovation, Organization and Strategy, Crios Working Paper n. 1. https://ssrn.com/abstract=2404344 (accessed: 01/06/2020). https://doi.org/10.2139/ssrn.2404344CoffanoM.TarasconiG.2014Crios—Patstat database: Sources, contents and access rulesCenter for Research on Innovation, Organization and StrategyCrios Working Paper n. 1. https://ssrn.com/abstract=2404344 (accessed: 01/06/2020). https://doi.org/10.2139/ssrn.2404344Search in Google Scholar

Derwent. (2007). World Patents Index—Derwent patentee codes, Revised edition 8. Thomson Corporation. Leuven Manual. ISBN: 0 901157 38 4. http://ips.clarivate.com/m/pdfs/mgr/patenteecodes.pdfDerwent2007World Patents Index—Derwent patentee codesRevised edition 8Thomson Corporation. Leuven ManualISBN: 0 901157 38 4. http://ips.clarivate.com/m/pdfs/mgr/patenteecodes.pdfSearch in Google Scholar

European Patent Office. (2019). Data catalog Patstat global. Versión 5.13. EPO Patstat customers. https://www.epo.orgEuropean Patent Office2019Data catalog Patstat global. Versión 5.13EPO Patstat customershttps://www.epo.orgSearch in Google Scholar

Gorraiz, J., Gumpenberger, C., & Wieland, M. (2011). “Galton 2011 revisited: A bibliometric journey in the footprints of a universal genius”. Scientometrics, 88(2), 627–652. https://doi.org/10.1007/s11192-011-0393-yGorraizJ.GumpenbergerC.WielandM.2011“Galton 2011 revisited: A bibliometric journey in the footprints of a universal genius”Scientometrics882627652https://doi.org/10.1007/s11192-011-0393-ySearch in Google Scholar

Guerrero-Bote, V.P., & Moya-Anegón, F. (2015). “Analysis of scientific production in food science from 2003 to 2013”. Journal of food science, 80(12), R2619–R2626. https://doi.org/10.1111/1750-3841.13108Guerrero-BoteV.P.Moya-AnegónF.2015“Analysis of scientific production in food science from 2003 to 2013”Journal of food science8012R2619R2626https://doi.org/10.1111/1750-3841.13108Search in Google Scholar

Guerrero-Bote, V.P., Sánchez-Jiménez, R., & Moya-Anegón, F. (2019). “The citation from patents to scientific output revisited: A new approach to Patstat / Scopus matching”. El profesional de la información, 28(4), e280401. https://doi.org/10.3145/epi.2019.jul.01Guerrero-BoteV.P.Sánchez-JiménezR.Moya-AnegónF.2019“The citation from patents to scientific output revisited: A new approach to Patstat / Scopus matching”El profesional de la información284e280401https://doi.org/10.3145/epi.2019.jul.01Search in Google Scholar

Hane, P.J. (2004). “Elsevier announces Scopus service”. Information today. http://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16494 (accessed: 01/06/2020).HaneP.J.2004“Elsevier announces Scopus service”Information todayhttp://newsbreaks.infotoday.com/nbreader.asp?ArticleID=16494 (accessed: 01/06/2020).Search in Google Scholar

Hinze, S., & Schmoch, U. (2004). Opening the black box. In: Moed, H.F., Glänzel, W., and Schmoch, U. (2004) (eds.). Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems. Dordrecht (the Netherlands): Kluwer Academic Publishers, 215–235.HinzeS.SchmochU.2004Opening the black boxIn:MoedH.F.GlänzelW.SchmochU.2004(eds.).Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systemsDordrecht (the Netherlands)Kluwer Academic Publishers215235Search in Google Scholar

Jacsó, P. (2011). “The h-index, h-core citation rate and the bibliometric profile of the Scopus database”. Online information review, 35(3), 492–501. https://doi.org/10.1108/14684521111151487JacsóP.2011“The h-index, h-core citation rate and the bibliometric profile of the Scopus database”Online information review353492501https://doi.org/10.1108/14684521111151487Search in Google Scholar

Jefferson, O.A., Jaffe, A., Ashton, D., Warren, B., Koellhofer, D., Dulleck, U., Bilder, G., Ballagh, A., Moe, J., DiCuccio, M., Ward, K., Bilder, G., Dolby, K., & Jefferson, R.A. (2018). “Mapping the global influence of published research on industry and innovation”. Nature biotechnology, 36(1), 31–39. https://doi.org/10.1038/nbt0818-772aJeffersonO.A.JaffeA.AshtonD.WarrenB.KoellhoferD.DulleckU.BilderG.BallaghA.MoeJ.DiCuccioM.WardK.BilderG.DolbyK.JeffersonR.A.2018“Mapping the global influence of published research on industry and innovation”Nature biotechnology3613139https://doi.org/10.1038/nbt0818-772aSearch in Google Scholar

Kang, B., & Tarasconi, G. (2016). “Patstat revisited: Suggestions for better usage”. World patent information, 46, 56–63. https://doi.org/10.1016/j.wpi.2016.06.001KangB.TarasconiG.2016“Patstat revisited: Suggestions for better usage”World patent information465663https://doi.org/10.1016/j.wpi.2016.06.001Search in Google Scholar

Leydesdorff, L., Moya Anegón, F., & Guerrero-Bote, V.P. (2010). “Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI”. Journal of the American Society for Information Science and Technology, 61(2), 352–369. https://doi.org/10.1002/asi.21250LeydesdorffL.Moya AnegónF.Guerrero-BoteV.P.2010“Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI”Journal of the American Society for Information Science and Technology612352369https://doi.org/10.1002/asi.21250Search in Google Scholar

Lissoni, F. (2012). “Academic patenting in Europe: An overview of recent research and new perspectives”. World patent information, 34(3), 197–205. https://doi.org/10.1016/j.wpi.2012.03.002LissoniF.2012“Academic patenting in Europe: An overview of recent research and new perspectives”World patent information343197205https://doi.org/10.1016/j.wpi.2012.03.002Search in Google Scholar

Lotti, F., & Marin, G. (2013). “Matching of Patstat applications to AIDA firms: Discussion of the methodology and results”. Bank of Italy occasional paper, n. 166. https://ssrn.com/abstract=2283111 (accessed: 01/06/2020). https://doi.org/10.2139/ssrn.2283111LottiF.MarinG.2013“Matching of Patstat applications to AIDA firms: Discussion of the methodology and results”Bank of Italy occasional paper, n. 166https://ssrn.com/abstract=2283111 (accessed: 01/06/2020). https://doi.org/10.2139/ssrn.2283111Search in Google Scholar

Lundberg, J. (2007). “Lifting the crown—citation z-score”. Journal of Informetrics, 1(2), 145–154. https://doi.org/10.1016/j.joi.2006.09.007LundbergJ.2007“Lifting the crown—citation z-score”Journal of Informetrics12145154https://doi.org/10.1016/j.joi.2006.09.007Search in Google Scholar

Magerman, T., Van-Looy, B., & Song, X.Y. (2006). Data production methods for harmonized patent statistics: Patentee name standardization. Technical report, K.U. Leuven. https://ec.europa.eu/eurostat/documents/3888793/5836029/KS-AV-06-002-EN.PDF (accessed: 01/06/2020).MagermanT.Van-LooyB.SongX.Y.2006Data production methods for harmonized patent statistics: Patentee name standardizationTechnical report, K.U. Leuven. https://ec.europa.eu/eurostat/documents/3888793/5836029/KS-AV-06-002-EN.PDF (accessed: 01/06/2020).Search in Google Scholar

Maraut, S., & Martínez, C. (2014). “Identifying author–inventors from Spain: Methods and a first insight into results”. Scientometrics, 101(1), 445–476. https://doi.org/10.1007/s11192-014-1409-1MarautS.MartínezC.2014“Identifying author–inventors from Spain: Methods and a first insight into results”Scientometrics1011445476https://doi.org/10.1007/s11192-014-1409-1Search in Google Scholar

Martínez, C. (2011). “Patent families: When do different definitions really matter?”, Scientometrics, v. 86, pp. 39–63. https://doi.org/10.1007/s11192-010-0251-3MartínezC.2011“Patent families: When do different definitions really matter?”Scientometrics863963https://doi.org/10.1007/s11192-010-0251-3Search in Google Scholar

Moya-Anegón, F., Chinchilla-Rodríguez, Z., Vargas-Quesada, B., Corera-Álvarez, E., Muñoz-Fernández, F.J., González-Molina, A., & Herrero-Solana, V. (2007). “Coverage analysis of Scopus: A journal metric approach”. Scientometrics, 73(1), 53–78. https://doi.org/10.1007/s11192-007-1681-4Moya-AnegónF.Chinchilla-RodríguezZ.Vargas-QuesadaB.Corera-ÁlvarezE.Muñoz-FernándezF.J.González-MolinaA.Herrero-SolanaV.2007“Coverage analysis of Scopus: A journal metric approach”Scientometrics7315378https://doi.org/10.1007/s11192-007-1681-4Search in Google Scholar

Moya-Anegón, F., Guerrero-Bote, V.P., López-Illescas, C., & Moed, H.F. (2018). “Statistical relationships between corresponding authorship, international co-authorship and citation impact of national research systems”. Journal of informetrics, 12(4), 1251–1262. https://doi.org/10.1016/j.joi.2018.10.004Moya-AnegónF.Guerrero-BoteV.P.López-IllescasC.MoedH.F.2018“Statistical relationships between corresponding authorship, international co-authorship and citation impact of national research systems”Journal of informetrics12412511262https://doi.org/10.1016/j.joi.2018.10.004Search in Google Scholar

Moya-Anegón, F., López-Illescas, C., Guerrero-Bote, V.P., & Moed, H.F. (2020). The citation impact of social sciences and humanities upon patentable technology. Scientometrics, to be published.Moya-AnegónF.López-IllescasC.Guerrero-BoteV.P.MoedH.F.2020The citation impact of social sciences and humanities upon patentable technologyScientometricsto be published.Search in Google Scholar

Narin, F., & Olivastro, D. (1992). Status report: Linkage between technology and science. Research Policy, 21, 237–249.NarinF.OlivastroD.1992Status report: Linkage between technology and scienceResearch Policy21237249Search in Google Scholar

Narin, F., Hamilton, K.S., & Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy, 26, 317–330.NarinF.HamiltonK.S.OlivastroD.1997The increasing linkage between US technology and public scienceResearch Policy26317330Search in Google Scholar

Organisation for Economic Co-operation and Development. (2009). OECD Patent Statistics Manual, OECD, Paris. https://www.oecd.org/sti/inno/oecdpatentstatisticsmanual.htmOrganisation for Economic Co-operation and Development2009OECD Patent Statistics ManualOECDParishttps://www.oecd.org/sti/inno/oecdpatentstatisticsmanual.htmSearch in Google Scholar

Pickering, B. (2004). “Elsevier prepares Scopus to rival ISI Web of science”. Information world review, n. 8.PickeringB.2004“Elsevier prepares Scopus to rival ISI Web of science”Information world review8Search in Google Scholar

Poege, F., Harhoff, D., Gaessler, F., & Baruffaldi, S. (2019). Science quality and the value of inventions. Science advances, 5(12), easy7323.PoegeF.HarhoffD.GaesslerF.BaruffaldiS.2019Science quality and the value of inventionsScience advances512easy7323Search in Google Scholar

Price, D.J.D (1965). Networks of scientific papers. Science, 149, 510–515.PriceD.J.D1965Networks of scientific papersScience149510515Search in Google Scholar

Raffo, J.D., & Lhuillery, S. (2009). “How to play the ‘Names game’: Patent retrieval comparing different heuristics”. Research policy, 38(10), 1617–1627. https://doi.org/10.2139/ssrn.1441172RaffoJ.D.LhuilleryS.2009“How to play the ‘Names game’: Patent retrieval comparing different heuristics”Research policy381016171627https://doi.org/10.2139/ssrn.1441172Search in Google Scholar

Schmoch, U., & Khan, M. (2019). Methodological challenges for creating patent indicators. In: Glanzel, W., Moed, H.F., Schmoch, U., & Thelwall, M. (eds.), Springer Handbook of Science and Technology Indicators. Cham (Switzerland): Springer Nature, 907–927.SchmochU.KhanM.2019Methodological challenges for creating patent indicatorsIn:GlanzelW.MoedH.F.SchmochU.ThelwallM.(eds.),Springer Handbook of Science and Technology IndicatorsCham (Switzerland)Springer Nature907927Search in Google Scholar

Schoen, A., Heinisch, D., & Buenstorf, G. (2014). “Playing the ‘Name game’ to identify academic patents in Germany”. Scientometrics, 101(1), 527–545. https://doi.org/10.1007/s11192-014-1400-xSchoenA.HeinischD.BuenstorfG.2014“Playing the ‘Name game’ to identify academic patents in Germany”Scientometrics1011527545https://doi.org/10.1007/s11192-014-1400-xSearch in Google Scholar

SCImago Research Group. (2010). SIR Methodology: General Considerations. https://www.scimagoir.com/methodology.php (accessed: 01/06/2020).SCImago Research Group2010SIR Methodology: General Considerationshttps://www.scimagoir.com/methodology.php (accessed: 01/06/2020).Search in Google Scholar

Thoma, G., & Torrisi, S. (2007). Creating powerful indicators for innovation studies with approximate matching algorithms. A test based on Patstat and Amadeus databases (No. 211). KITeS, Centre for Knowledge, Internationalization and Technology Studies, Università Bocconi, Milano, Italy. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.573.8107&rep=rep1&type=pdf (accessed: 01/06/2020).ThomaG.TorrisiS.2007Creating powerful indicators for innovation studies with approximate matching algorithms. A test based on Patstat and Amadeus databases (No. 211)KITeS, Centre for Knowledge, Internationalization and Technology Studies, Università BocconiMilano, Italyhttp://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.573.8107&rep=rep1&type=pdf (accessed: 01/06/2020).Search in Google Scholar

Van Raan, A.F.J. (2017). Patent Citations Analysis and Its Value in Research Evaluation: A Review and a New Approach to Map Technology-relevant Research. Journal of Data and Information Science, 2, 13–50.Van RaanA.F.J.2017Patent Citations Analysis and Its Value in Research Evaluation: A Review and a New Approach to Map Technology-relevant ResearchJournal of Data and Information Science21350Search in Google Scholar

Veugelers, R., & Wang, J. (2019). Scientific novelty and technological impact. Research Policy, 48(6), 1362–1372.VeugelersR.WangJ.2019Scientific novelty and technological impactResearch Policy48613621372Search in Google Scholar

Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., & van Raan, A.F. (2011). “Towards a new crown indicator: Some theoretical considerations”. Journal of informetrics, 5(1), 37–47. https://doi.org/10.1016/j.joi.2010.08.001WaltmanL.van EckN.J.van LeeuwenT.N.VisserM.S.van RaanA.F.2011“Towards a new crown indicator: Some theoretical considerations”Journal of informetrics513747https://doi.org/10.1016/j.joi.2010.08.001Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo