Forgotten Knowledge: Examining the Citational Amnesia in NLP

Abstract

Citing papers is the primary method through which modern scientific writing discusses and builds on past work. Collectively, citing a diverse set of papers (in time and area of study) is an indicator of how widely the community is reading. Yet there is little work looking at broad temporal patterns of citation. This work, systematically and empirically examines: How far back in time do we tend to go to cite papers? How has that changed over time, and what factors correlate with this citational attention/amnesia? We chose NLP as our domain of interest, and analyzed ∼71.5K papers to show and quantify several key trends in citation. Notably, ∼62% of cited papers are from the immediate five years prior to publication, whereas only ∼17% are more than ten years old. Furthermore, we show that the median age and age diversity of cited papers was steadily increasing from 1990 to 2014, but since then the trend has reversed, and current NLP papers have an all-time low temporal citation diversity. Finally, we show that unlike the 1990s, the highly cited papers in the last decade were also papers with the least citation diversity; likely contributing to the intense (and arguably harmful) recency focus. Code, data, and a demo are available at the project homepage.