Professional Documents
Culture Documents
An Indepth Analysis of Government Funding and International
An Indepth Analysis of Government Funding and International
net/publication/342378359
CITATIONS READS
0 226
3 authors:
Xiaozan Lyu
Zhejiang University
13 PUBLICATIONS 81 CITATIONS
SEE PROFILE
All content following this page was uploaded by Xiaojing Cai on 21 July 2020.
1
Zhejiang University, School of Public Affairs, Dept of Information Resources Management, No. 866
Yuhangtang Road, 310059 Hangzhou (China)
2
Leiden University, Central for Science and Technology Studies (CWTS), Faculty of Social and Behavioral
Sciences, Leiden University, 2300 AX Leiden (Netherlands)
Abstract
Based on publications indexed in the Web of Science, the current study focused on internationally collaborated
publications and publications acknowledging government funding of developing and developed countries
including China, the US, Germany, the Netherlands, South Africa, and Brazil. Specific focus is placed on
national funding agencies (i.e., focal agencies) supporting competitive research in science. The focal agencies of
the six countries include the National Natural Science Foundation of China (NSFC), the US National Science
Foundation (NSF), German Research Foundation (DFG), the Netherlands Organization for Scientific Research
(NWO), National Research Foundation of South Africa (NRF), and National Council for Scientific and
Technological Development of Brazil (CNPq). Results show that different countries vary in arrangement of
government funding sources in support of competitive research projects. China and Brazil are centralized to the
NSFC and CNPq, whereas the remaining four countries have relatively decentralized sources. The six focal
national funding agencies of the current study are more efficient than non-focal agencies in raising citation
impact, with the NWO, NSF, and NSFC perform better than non-focal agencies of corresponding countries.
International collaboration improves citation impact, with the developing countries benefit more. Authorship
(first or corresponding authorship) also plays a role in international collaboration. Collaboration led by authors
from developed countries has a positive effect whereas that led by authors from developing countries usually
have negative effects on citation impact. Interaction between funding and collaboration may have results
different from those when these two factors are considered separately, which suggests being cautious while
discussing effects of either of the two factors.
Article Highlights
⚫ Different countries vary in arranging government funding of competitive research. China and Brazil
have centralized funding sources to the NSFC and the CNPq, whereas thoses of the other four
countries are relatively decentralized.
⚫ The share of funded papers in all countries continues to increase, but that funded by the focal agencies
differs: that of the NSFC continues to grow, that of South Africa grows slightly, and those of the
remaining countries decline.
⚫ Funding helps improve citation impact. Among the six focal agencies, the NWO, NSF and NSFC
perform better in raising citation impact compared with domestic non-focal agencies.
⚫ International collaboration improves citation impact of developing countries. Collaborating with the
developed countries, especially with the US works best. Collaboration led by authors from the
developing countries may have negative effects in terms of citation impact of publications.
⚫ Interaction between funding and collaboration may have results different from those when these two
factors are considered separately.
Corresponding author
Ping Zhou, Zhejiang University, School of Public Affairs, Dept of Information Resources Management, No. 866
Yuhangtang Road, 310059 Hangzhou (China). Email: pingzhou@zju.edu.cn
Government funding plays a significant role in the development of science, and has attracted extensive interests
from academic community. Positive effects of government funding on raising citation impact of publications (e.g.,
Yan, Wu, & Song, 2018) and breakthrough inventions (Corredoira, Goldfarb, & Shi, 2018) have been found.
Variations in terms of funding efficiency may exist between different funding systems. Among many factors,
national research evaluation systems (Sandström & Van Besselaar, 2016), academic freedom, and university
stratification play significant roles in affecting funding efficiency (Sandström & Van den Besselaar, 2018). In the
current era that scientific discovery increasingly relies on wide-spread collaboration (Bozeman & Corley, 2004;
Choi, Yang, & Park, 2015; Cimini, Zaccaria, & Gabrielli, 2016; Wagner, 2018) including international
collaboration (Frame & Carpenter, 1979; Wagner, Park, & Leydesdorff, 2015), international collaboration has
become a hot topic in recent years (Chen, Zhang, & Fu, 2019; D’Ippolito & Rüling, 2019). Research topics vary
from the roles of international collaboration in publication productivity and citation impact (Bozeman, Fay, &
Slade, 2013; Lee & Bozeman, 2005; Van Raan, 1998), network position of specific countries (Adams, 2012; Guan,
Yan, & Zhang, 2017; Todeva & Knoke, 2005; Wang, Wang, & Philipsen, 2017; Zhao & Guan, 2011), to
collaboration patterns (Leydesdorff & Wagner, 2008; Todeva & Knoke, 2005). The effect of international
collaboration on raising citation impact of publications varies significantly among countries and is dependent on
collaborating partners to a large extent (De Moya-Anegon, Guerrero-Bote, Lopez-Illescas, & Moed, 2018;
Guerrero Bote, Olmeda-Gómez, & de Moya-Anegón, 2013; Lancho-Barrantes, Guerrero-Bote, & de Moya-
Anegón, 2013; Sud & Thelwall, 2016). Collaborating with the US especially when the US researchers serve as
corresponding authors would benefit most (De Moya-Anegon et al., 2018; Guerrero Bote et al., 2013; Moya-
Anegón, Guerrero-Bote, Bornmann, & Moed, 2013).
Both funding and international collaboration play vital roles in scientific research but were treated in isolation in
many studies. One typical perspective is on the role of funding in facilitating international collaboration, and with
no exception, positive effect has been confirmed (Cimini et al., 2016; Clark & Llorens, 2012; Liu, Liang, Tuuli,
& Chan, 2018; Ubfal & Maffioli, 2011). Another popular perspective is on whether international collaboration
facilitates access to funding supports (Zhou & Tian, 2014). The positive effect of funding and international
collaboration on citation impact have been proven when the two elements (funding and international collaboration)
were viewed in isolation (Leydesdorff, Wagner, & Bornmann, 2019; Zhou, Zhong, & Yu, 2013), otherwise, the
effect of government funding tends to be a small adverse (Leydesdorff, Bornmann, & Wagner, 2019).
Given that funding and international collaboration may interact with each other and thus produce a more complex
effect on citation impact, we decide to explore further on this topic, and presented the results at the ISSI 2019
(Zhou, Cai, Xiong, & Lyu, 2019).The current version is an extension of the conference paper. More factors such
as collaboration size, funding sources, and research fields are taken into consideration. By focusing on publications
acknowledging government funding of six different countries, we hope to answer the following questions: (1)
What is the contribution of funded research to science? Does country variation exist? (2) Are there differences
among countries in the citation impact of their publications acknowledging support of national funding agencies?
(3) What is the role of international collaboration in different countries? Does country variation exist in the effect
3
Bibliometric data in 2009-2016 are extracted from the CWTS-licensed version of the Web of Science (WoS)
database of Clarivate. Six countries including China, Brazil, South Africa, Germany, the Netherlands, and the US
are selected to illustrate the situations in both developing and developed countries. The national funding agencies
supporting basic research of the six countries (Table 1) will be a focus, and thus will be named as focal agencies
for convenience, and those not supported by the focal agencies are named as non-focal agencies. Funded
publications are harvested from CWTS funding organization database originated from the WoS index fields FO
and FT. Publications acknowledging support of a focal agency but none of the authors from the corresponding
country are excluded to avoid possible disturbance of results.
Field classification is based on CWTS 35 subject categories. Because not all scientific fields are supported by the
focal agencies of the current study, we only cover subject categories that are main supporting areas of the six focal
agencies. Thus, 22 subject categories1 remain. Since the current study focuses on journal publications, only those
with journal papers as a main type of research outcome will be covered. Although highly productive in journal
publications, the area Clinical Medicine is not covered because it is not the major area supported by the National
Science Foundation (NSF_US). In the end, six subject categories including Basic Life Sciences, Biological
Sciences, Chemistry & Chemical Engineering, Environmental Sciences & Technology, Mathematics as well as
Physics & Materials Science remain.
Two sets of indicators measuring publication productivity and citation impact are used. For productivity, we apply
the percentage of funded publications. The Mean Normalized Citation Scores (MNCS) proposed by CWTS and
percentage of top-1% highly cited publications are used to measure the average citation impact of a publication
category (see Table 2) under variable citation window.
1
Agriculture and Food Science, Astronomy and Astrophysics, Basic Life Sciences, Basic Medical Sciences,
Biological Sciences, Biomedical sciences, Chemistry and Chemical Engineering, Civil Engineering and
Construction, Clinical Medicine, Computer Sciences, Earth Sciences and Technology, Electrical Engineering
and Telecommunication, Energy Science and Technology, Environmental Sciences and Technology, General
and Industrial Engineering, Health Sciences, Instruments and Instrumentation, Mathematics, Mechanical
Engineering and Aerospace, Multidisciplinary Journals, Physics and Materials Science, Statistical Sciences.
4
To analyze the effect of different factors on citation impact, publications are classified into different categories as
illustrated in Table 2.
Results
Both descriptive and regression results will be illustrated in this section. Ordinary Least Square (OLS) regression
with robust standard errors is used to verify the effects of funding and international collaboration on citation impact
when they are considered alone and together.
As expected, the results further prove the importance of funding supports in national publication production, but
country variations exist. Around 80% of publications of the six countries have acknowledged grant support. With
over 87% of funded publications, China is most prominent. Situations in the remaining five countries are similar
(70-77%). In terms of percentage of publications acknowledging grant of the focal agencies in the total national
funded publications (i.e., % (PFF/PFA)), that of CNPq_BR and NSFC_CN is the highest and with similar value
(71%). In other words, about 71% of funded publications of Brazil and China are supported by the corresponding
focal agencies (i.e., CNPq_BR and NSFC_CN). The percentage contribution of NSF_US to the US funded
publications is the least (24%), whereas that of DFG_DE, NRF_ZA, and NWO_NL are respectively 40%, 33%
and 32% (Fig 1a).
60
%
40
20
0
BR CN DE ZA NL US
With the highest percentage of funded publications, China far exceeds the other five countries. The six countries
have one thing in common: They all had a growing share of funded publications in national total until 2015 when
the growth momentum started to change – five countries (i.e., China, the US, Germany, The Netherlands, and
South Africa) stopped growing, leaving Brazil the only one keeping the growth momentum (Fig 1b). With respect
to the share of a focal agency in total funded publications (PFF in PFA) of a country, that of NSFC_CN has kept
growing, that of NRF_ZA has increased slightly, and those of the remaining four countries have decreased (Fig
1c). The growing share of NSFC_CN in funded publications of China implies the critical role of NSFC_CN in
Citation impact measured by MNCS is, however, a different landscape. The developed countries perform better
than the developing ones. When comparison is done among each type of publications, the Netherlands performs
best followed by the US and Germany. Funded publications including those of PFF and PFA-without-PFF perform
better than unfunded ones (PnotFA). Publications funded by the focal agencies (PFF) perform better than those
funded by non-focal agencies (PFA-without-PFF) in the Netherlands (NWO_NL), the US (NSF_US), and China
(NSFC_CN), whereas the opposite is true in Germany (DFG_DE), Brazil (CNPq_BR), and South Africa
(NRF_ZA). In the situation that citation impact of unfunded publications of China is remarkably lower than that
of world average and is even lower than that of South Arica, that of publications funded by NSFC_CN reach a
level higher than that of world average, and to a level higher than that of funded publications of South Africa (Fig
2a).
a. MNCS b. TOP-1%
2.0 3.0
PFF PFA-without-PFF PnotFA PFF PFA-without-PFF PnotFA
2.5
1.5
2.0
%
1.0 1.5
1.0
0.5
0.5
0.0 0.0
NL US DE CN ZA BR NL US DE CN ZA BR
Similar situation happens in terms of share of top-1% highly cited publications for the three categories (i.e., PFF,
PFA-without-PFF, and PnotFA): the developed countries perform better than the developing ones. When
comparison is done among each category of publications, the Netherlands performs best followed by the US and
Germany. Funded publications including both PFF and PFA-without-PFF perform better than unfunded ones
(PnotFA). Publications funded by the focal agencies (PFF) perform better than those funded by non-focal agencies
(PFA-without-PFF) in the Netherlands (NWO_NL), the US (NSF_US), and China (NSFC_CN), whereas the
opposite is true in Germany (DFG_DE), Brazil (CNPq_BR), and South Africa (NRF_ZA). In the situation that the
percentage of top-1% highly cited publications of unfunded Chinese publications is lower than 1% and is even
lower than that of South Africa, NSFC_CN funding has raised the indicator of Chinese publications to a level close
to that of German DFG_DE funded publications (Fig 2b).
% (PFA/all)
100
90
80
%
70
60
50
CN BR US DE ZA NL
Basic Life Sciences Biological Sciences
Chemistry & Chemical Engineering Environmental Sciences & Technology
Mathematics Physics & Materials Science
With regard to the ratio of focal-agency funded to total funded publications (PFF/PFA) of a country, the six
countries can be divided into two groups (Fig 4). One group has high percentage of PFF/PFA (61-80%) and
consists of China and Brazil. The other group is composed of the remaining four countries with significantly lower
percentage of PFF/PFA (11-53%). The NSFC_CN or CNPq_BR take the absolute lead in supporting basic
research and contribute most to funded publications of the two countries. Although Mathematics has the least
percentage of funded publications in the US, Germany, the Netherlands, and South Africa (Fig 3), but has the
highest percentage of PFF/PFA among the six fields in the US (Fig 4).
45
30
15
0
CN BR US DE ZA NL
Basic Life Sciences Biological Sciences
Chemistry & Chemical Engineering Environmental Sciences & Technology
Mathematics Physics & Materials Science
Fig 4. Percentage share of publications supported by the focal agencies in the total funded publications of a
country.
In the following, we will analyze citation impact measured by MNCS of each of the six countries in the six subject
areas. The rank order follows that exhibited in Fig 2 in most cases. The developed countries take the lead and
China frequently stands between the developed and developing countries. the Netherlands takes the lead in the six
areas except for Mathematics and Chemistry & Chemical Engineering where the US is the leader. The second
positions in the other four areas are taken by the US. Germany takes the third position in five areas except for
Chemistry & Chemical Engineering in which the third position is taken by China. In Physics & Materials Science,
however, South Africa performs better than China (Fig 5).
Citation impact of each of the three sets of publications (PFF, PFA-without-PFF, and PnotFA) varies not only
within a country but also in different subject areas. In Basic Life Sciences, the Netherlands performs the best among
the six countries. As for citation impact of the three publication categories of a specific country, large variations
exist. NWO_NL-funded publications perform better than non-NWO_NL-funded ones. Similarly, NSF_US-funded
publications outperform those not funded by NSF_US. The opposite is true for those funded by DFG_DE,
NRF_ZA, and CNPq_BR. There is no clear difference between publications funded by NSFC_CN and non-
NSFC_CN. Funded publications perform better than unfunded ones in the Netherlands, the US, Germany, and
China. It is, however, not the case in South Africa and Brazil: Citation impacts of publications funded by NRF_ZA
or CNPq_BR are lower than those of unfunded, which is out of expectation (Fig 5).
In Biological Sciences, the Netherlands takes the lead again. NWO_NL-funded publications perform better than
those funded by non-NWO_NL agencies. NSF_US-funded publications significantly outperform non-NSF_US-
funded ones. Publications funded by the focal agencies of the remaining four countries perform poorer than those
not funded by the focal agencies. In Chemistry & Chemical Engineering, the US takes the lead in funded
publications. The focal agencies perform better than non-focal agencies in the US, the Netherlands and China. The
opposite is true in the other three countries (Germany, South Africa, and Brazil).
10
Publications funded by the NSF_US takes the lead and outperform those not funded by the NSF_US in
Mathematics. Publications funded by non-focal agencies in Germany, the Netherlands, and South Africa perform
better than those funded by DFG_DE, NWO_NL, and NRF_ZA. NSFC_CN-funded publications perform slightly
better than those not funded by the NSFC_CN. South Africa again is out of expectation in the poorer performance
of NRF_ZA-funded publications.
In Physics & Materials Science, the Netherlands takes the lead again. Publications funded by the focal agencies in
the Netherlands (NWO_NL), the US (NSF_US), South Africa (NRF_ZA), and China (NSFC_CN) perform better
than those not funded by the focal agencies. There is no clear difference for publications funded or unfunded by
DFG_DE or not. Brazil is the only country that non-CNPq_BR-funded publications even perform better than those
funded by CNPq_BR (Fig 5).
Regression analysis
In addition to funding support, many other factors may affect citation impact. By applying the Ordinary Least
Squares (OLS) regression, we hope to find if there is a positive or negative relation between citation impact and
other variables. In the regression, the Normalized Citation Score (NCS) of publications is the dependent variable
and publication categories (PnotFA, PFA-without-PFF, PFF) are independent variables. Dummy variables include
international collaboration (IntNatl), first authorship, corresponding authorship, as well as collaborations with the
US, G6 or non-G7 countries (i.e., Collaborating with the US, Collaborating with G6, and Collaborating with non-
G7). Funding support and international collaboration may interact with each other. To clarify if such interaction
exists, we use two sets of regression models. Model A (i.e., 1a, 2a, 3a, 4a, 5a, and 6a) does not considers such
interaction and Model B (i.e., 1b, 2b, 3b, 4b, 5b, and 6b) does. Other factors, such as publication year (Length of
years), number of references (log_refs), number of authors (log_au), number of institutions (log_ins), number of
countries (log_country), publication types (doc_article, doc_review, doc_letter) and scientific fields, may also
affect citation impact, and thus are set as controlled variables. Stepwise method is used in the regression so as to
avoid multicollinearity among the independent variables (Table 3).
The regression results of Models A and B are, in general, consistent with the results in the descriptive analysis.
Funding supports, regardless of focal or non-focal agencies of any of the six countries except for South Africa,
have positive effect on citation impact of publications, although such effects may vary among countries. The
higher-level effect of NSF_US, NSFC_CN, and NWO_NL, and lower-level effect of DFG_DE in comparison with
non-focal agencies, have been confirmed again (Table 3).
11
12
With regard to the role of international collaboration in raising citation impact, the developing countries benefit,
whereas the developed countries do not. With 11.3% (e0.107) increase of citation impact by collaborating with
foreign partners, China has benefited the most. Brazil and South Africa are also beneficiary with citation impact
being increased by 10.3% and 3.9% (e0.098 and e0.038) respectively. On the contrary, citation impacts of
internationally collaborated publications of the US has been lowered by about 1% (e -0.01). International
collaboration does not have significant impact, either positive or negative, on publications of German and Dutch
publications (Model A in Table 3).
Funding support has positive effect on raising citation impact, whereas the effect of international collaboration can
be positive, negative, or neutral to publications of different countries. What may happen when both funding and
international collaboration play roles together? The regression Model B may answer this question. China and
Brazil share some common points. The citation-raising effects of international collaboration on publications
funded by either the NSFC_CN or CNPq_BR are not as high as that of unfunded. It is also valid to say that either
the NSFC_CN or CNPq_BR is less effective in raising citation impact of internationally collaborated publications
than that of publications not internationally collaborated. Such kind of difference does not exist between
international collaboration and non-NSFC_CN funding of Chinese publications. For Brazilian case, the
international collaboration sees an increased effect of non-CNPq_BR funding and decreased effect of CNPq_BR
funding on the citation impact. The uniqueness of South Africa appears again when international collaboration and
funding are concerned separately, either does not have significant effect. When interactions of the two factors are
considered, the effects of non-NRF_ZA support on international collaboration or that of international collaboration
on publications of non-NRF_ZA funding are positive, which further confirm the relative inefficiency of NRF_ZA
in improving citation impact. The positive effect of NSF_US or DFG_DE on international collaboration is very
much impressive, especially at the situation that the overall effects of international collaboration are negative in
the US and Germany. This may imply that both the NSF_US and DFG_DE have high evaluation criterion for
international collaboration so that high-quality research can be ensured. No significant interaction between
international collaboration and funding is observed in the Netherlands.
13
14
The fact that funded publications have higher citation impact may have two implications: On the one hand, funding
helps increase citation impact, and on the other hand, high citation impact increases the possibility of being funded.
Thus, it is safer to say that there exists statistical effect between funding and citation impact. Such effect varies
among countries. High funding rates of publications do not guarantee high citation impact. When citation impact
is concerned, the developed countries (i.e., the Netherlands, US, and Germany) perform better than developing
countries (i.e., South Africa, China, and Brazil), and the Netherlands performs the best. With regard to the effect
of focal versus non-focal agencies, the former does not always perform better than the latter in raising citation
impact. In most cases, the NWO_NL, NSF_US, and NSFC_CN perform better than non-focal agencies of the
corresponding countries, whereas the opposite is true for DFG_DE, NRF_ZA, and CNPq_BR. China stands
between the developed and the developing countries except in the following two subject areas: the first one is
Chemistry & Chemical Engineering where China is ahead of Germany and is ahead of the Netherlands in
Mathematics; The second area is Physics & Materials Science in which China even falls behind South Africa.
Similar to the situation when all publications are considered as a whole, the effect on improving citation impact of
some focal agencies is better than non-focal agencies, which applies to the NWO_NL, NSF_US, and NSFC_CN.
The opposite is true for the DFG_DE, NRF_ZA, and CNPq_BR.
Both descriptive and OLS regression analyses have reached the same conclusions that funding supports can help
raise citation impact. Variations exist in terms of citation-raising effects among different funding sources. Some
focal agencies (NWO_NL, NSF_US, and NSFC_CN) show higher effect than non-focal agencies, whereas some
(DFG_DE, NRF_ZA, and CNPq_BR) have lower effect. The situation in South Africa is the most unique – there
is no significant change of citation impact of publications funded by NRF_ZA but non-NRF_ZA funding does
have such effect. The above comparison is between focal and non-focal agencies within a country. For example,
when we say that NSFC_CN is better than non-focal-agency funding, the comparison is made within publications
of China. Similarly, the effect of DFG_DE is not as good as that of non-focal agencies, and the comparison is also
within publications of Germany. It is not correct to say that the NSFC_CN performs better than the DFG_DE
15
The positive role of international collaboration has been confirmed by many studies (e.g. De Moya-Anegon et al.,
2018; Zhou & Bornmann, 2014). The most representative one is that of De Moya-Anegon et. al (2018) because of
the wide coverage of data – all publications of all countries indexed in Scopus in 2003-2015 were analyzed. In
addition to proving the positive role of international collaboration, the authors found a negative role of
corresponding authorship. These conclusions are true from an overall perspective. If we investigate further on the
economic position of an international partner (a country), we may get more interesting results. Developing
countries benefit more than developed countries in international collaboration. Collaborating with developed
countries especially with the US can be a first option in choosing international partners.
International collaboration is important to promote the development of science. To the developed countries,
international collaboration should still be encouraged although citation impact of publications may be negatively
affected. The value of international collaboration is not just citation impact, it can promote academic
communication, enhance research capabilities, and expand academic network of researchers from developing
countries. In addition to contributing their talents, scholars from developing countries can make up for the shortage
of human resources faced by research projects of developed countries. The negative effect of international
collaboration of the current study are statistical results. For individual collaboration, however, it is more important
in selecting collaboration partners by their research qualification rather than by their country background.
Establishing appropriate project evaluation standards is essential to ensure good funding results. Such standards
can be responded by the fact that international collaboration funded by the NSF_US and German DFG_DE have
positive effects, whereas that of China’s NSFC_CN and Brazilian CNPq are less effective on citation impact.
References
Abramo, G., D’Angelo, A. C., & Murgia, G. (2017). The relationship among research productivity, research
collaboration, and their determinants. Journal of Informetrics, 11(4).
https://doi.org/10.1016/j.joi.2017.09.007
Adams, J. (2012). Collaborations: The rise of research networks. Nature. https://doi.org/10.1038/490335a
Bozeman, B., & Corley, E. (2004). Scientists’ collaboration strategies: Implications for scientific and technical
human capital. Research Policy, 33(4), 599–616. https://doi.org/10.1016/j.respol.2004.01.008
Bozeman, B., Fay, D., & Slade, C. P. (2013). Research collaboration in universities and academic
entrepreneurship: The-state-of-the-art. Journal of Technology Transfer. https://doi.org/10.1007/s10961-
012-9281-8
Chen, K., Zhang, Y., & Fu, X. (2019). International research collaboration: An emerging domain of innovation
studies? Research Policy, 48(1), 149–168. https://doi.org/10.1016/j.respol.2018.08.005
Choi, S., Yang, J. S. W., & Park, H. W. (2015). The triple helix and international collaboration in science.
Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.23165
16
17
18