Inflated impact factors? The true impact of evolutionary papers in non-evolutionary journals

Abstract

Amongst the numerous problems associated with the use of impact factors as a measure of quality are the systematic differences in impact factors that exist among scientific fields. While in theory this can be circumvented by limiting comparisons to journals within the same field, for a diverse and multidisciplinary field like evolutionary biology, in which the majority of papers are published in journals that publish both evolutionary and non-evolutionary papers, this is impossible. However, a journal's overall impact factor may well be a poor predictor for the impact of its evolutionary papers. The extremely high impact factors of some multidisciplinary journals, for example, are by many believed to be driven mostly by publications from other fields. Despite plenty of speculation, however, we know as yet very little about the true impact of evolutionary papers in journals not specifically classified as evolutionary. Here I present, for a wide range of journals, an analysis of the number of evolutionary papers they publish and their average impact. I show that there are large differences in impact among evolutionary and non-evolutionary papers within journals; while the impact of evolutionary papers published in multidisciplinary journals is substantially overestimated by their overall impact factor, the impact of evolutionary papers in many of the more specialized, non-evolutionary journals is significantly underestimated. This suggests that, for evolutionary biologists, publishing in high-impact multidisciplinary journals should not receive as much weight as it does now, while evolutionary papers in more narrowly defined journals are currently undervalued. Importantly, however, their ranking remains largely unaffected. While journal impact factors may thus indeed provide a meaningful qualitative measure of impact, a fair quantitative comparison requires a more sophisticated journal classification system, together with multiple field-specific impact statistics per journal. © 2007 Erik Postma.