The Fall of Scientometrics: Its Political Rise and Scientific Demise

Abstract

This paper is a critique of scientometrics as both a meta-science and a scientific policy within the context of the ongoing methodological crisis in science. Scientometrics emerged as a quantitative social ‘science of science’, under the assumption that citations reflect the value scientists attribute to others’ work and that their quantity serves as an objective measure of ‘scientific quality’. However, during the ‘replication crisis’ it became apparent that many widely cited statistical studies from various scientific fields could not be replicated; that the prevailing research practice of statistical hypothesis testing is riddled with methodological flaws and abuses; and also that falsified research continues to be cited unabated even after unsuccessful replication attempts. These issues prove that scientometrics cannot be justified as a science that objectively describes scientific value. Earlier critics argued that using quantification for the purpose of evaluating articles across different disciplines implies abstracting their content, and thus, the citation motives, which otherwise vary empirically. However, when it was introduced into science policy, scientometrics transformed into an applied science that could ignore these theoretical problems because the purpose of citation analysis was not the objective evaluation of scientific work. Starting from the fact that the spreading of the competitive market logic across all aspects of society is the essence of neoliberal political projects, this paper argues that scientometric indicators in science policy serve as a quasi-scientific technology for simulating and legitimizing competition among academic entities. In this way, they stimulate rational behavior with scarce resources among actors in the process of producing scientific ‘goods’ for the academic ‘market of ideas’. As a form of neoliberal management of society through numbers, evaluative scientometrics falls prey to Goodhart's Law. When ‘good numbers’ are set as the goal of an activity, actors focus on them at the expense of the integrity of the original activity. Scientists change and adapt their behavior, including the interpretation and application of methodological standards, to meet quantitative criteria and ensure their professional existence. This explains the current methodological and confidence crisis in science.