Web of Science: Showing a Bug Today That Can Mislead Scientific Research Output Prediction

Abstract

As in all domains of human activity, economic issues and the increase of people working in scientific research have altered the way scientific production is evaluated as well as the objectives for performing the evaluation. The h index was introduced in 2005 by J. E. Hirsch as an indicator for the measurement of individual scientific output not only in terms of quantity but also in terms of quality, and its use has since spread throughout the world. In 2007, Hirsch proposed its adoption as the best way to predict future scientific achievement and, consequently, as a useful guide for investment in research and for institutions when hiring members for their scientific staff. Since then, several authors have also been using the Thomson ISI Web of Science database to develop their proposals for evaluating research output. Here, using a software we have developed, we analyze more than 100,000 articles and show that a subtle flaw in Web of Science can inflate the results of information collected, therefore compromising the exactness and, consequently, the effectiveness of Hirsch’s proposal and its variations.