In an order released last week, China’s science and education ministries also say that institutions must not promote or recruit researchers solely on the basis of the number of papers they publish, or their citations. Researchers are welcoming the policy, but say that it could reduce the country’s competitiveness in science.
In China, one of the main indicators currently used to evaluate researchers, allocate funding and rank institutions is metrics collected by the Science Citation Index (SCI), a database of articles and citation records for more than 9,000 journals. Since 2009, articles in these journals written by authors from Chinese institutions increased from some 120,000 a year to 450,000 in 2019. Some institutions even pay researchers bonuses for publishing in them.
These practices have incentivized researchers to publish lots of papers at the expense of quality, says Jin Xuan, a chemical engineer at Loughborough University, UK. Evidence suggests that the focus on metrics has also driven a rise in inappropriate practices, such as researchers submitting plagiarized or fraudulent papers, or inappropriately citing their own or a colleague’s work to boost citations1.
The goal of the new policy is not to discourage Chinese researchers from publishing papers in SCI-listed journals, but to stop inappropriate publishing and citation practices, says Tang Li, a researcher of science and technology policy at Fudan University in Shanghai, China.
Xuan adds that the policy aligns well with global declarations, such as the San Francisco Declaration on Research Assessment, that aim to move away from an over-reliance on these types of metric in research appraisals and to limit perverse incentives that drive researchers to engage in questionable research practices.
Potentially fraudulent papers from authors in China have been in the spotlight over the past year2. Concerned researchers have been collating examples of hundreds of papers that contain apparently duplicated images and similar layouts3 , which journal publishers say they are investigating. The researchers suspect that these papers might have been produced to order to meet performance quotas.
New assessments
As part of the new policy, researcher assessments will now need to use indicators of the quality of research, such as how innovative the work is, whether it represents a significant scientific advance, or its contribution to solving important societal problems. These evaluations should also rely more heavily on the professional opinions of expert peers, and consider research in journals published in China, many of which are not listed in the SCI.
But Futao Huang, who studies higher-education policy at Hiroshima University, Japan, says it is not clear what exactly the new evaluation system will look like, because the ministry’s notices lack specific and practical recommendations for what should replace the current system.
Huang thinks the new measures could result in a drop in the number of low-quality or fraudulent papers, but might also trigger a decline in China’s total publications in indexed journals as researchers feel less pressure to publish to gain degrees, promotions or funding.
And fewer Chinese papers in indexed journals could affect the country’s research competitiveness, says Huang. International researchers might be less inclined to collaborate with Chinese academics without a publication record in these journals, and fewer papers could push Chinese universities lower down in international rankings, he says.
Xuan says the focus on assessing researchers on the basis of their work in Chinese journals is controversial because a lot of them publish in Mandarin, and the journals are unknown to scientists outside China. “This will, to some extent, isolate the Chinese researchers from the global research community,” he says.
Other scientists have raised concerns about the new assessments relying too heavily on peer reviews, which are subjective and could create conflicts of interest or place too much emphasis on personal relationships. Without transparent and consistent evaluation criteria for promoting researchers and allocating funding, there is a risk that researchers will not be evaluated on the basis of merit, says Tang.
The new measures will apply to graduate students and researchers at universities, research institutions and research hospitals in China. The ministries have asked research-intensive universities to revise their assessment policies by 31 July. Institutions that continue to incentivize scientists to publish papers in SCI journals will have funding for special projects suspended.
