The “Free lunches” index for assessing academics: a not entirely serious proposal

Abstract

Background: Evaluating academics is a challenge, and the use of indicators such as scientific impact (i.e. number of published papers and their citation rate) is complex and poorly validated. We propose a new indicator for academic medical research: the “Free lunches” index (fl-index), computed from the sum of gifts from the industry. The fl-index provides a direct and straightforward measure of industry investment consisting in regaling a clinical researcher with rewards like a leisurely meal in a Michelin-starred restaurant or a relaxing stay in a high-end resort hotel. Methods and findings: 3,936 French academics were included in this observational and satirical retrospective study using the French database registering gifts received by medical doctors and Web of Science, over the years 2014–2019. Pearson’s correlation coefficients explored the associations between the fl-index and in the h-index (the maximum number of published papers h that have each been cited at least h times) increase over the period 2014–2019. The diagnostic properties and optimal thresholds of the fl-index for detecting high scientific productivity were explored. High scientific productivity was defined as ranking in the top 25% scientists in terms of increase in the h-index. To detect possible differences according to medical disciplines, subgroup analyses were performed. The correlation coefficient between the fl-index and the increase in the h-index was 0.31 (95% CI 0.29 to 0.34). The optimal threshold was 7,700 € for the fl-index, giving a sensitivity of 65% (95% CI 61 to 67%), a specificity of 59% (95% CI 57 to 61%). However, there were considerable differences across medical disciplines, with correlations ranging from 0.12 (Morphology and morphogenesis) to 0.51 (Internal medicine, geriatrics, general surgery and general medicine), and the median fl-index ranging from 37 € (Public health, environment and society) to 30 404 € (Cardiorespiratory and vascular pathologies). Importantly, the highest correlations and values for the fl-index were observed for clinical disciplines. Conclusions: Overall, the correlation between the fl-index and an increase in the h-index was modest so that the fl-index cannot be used as a surrogate for academic success as gauged by productivity-based metrics. However, future residents could use these results to complement the usual metrics in order to choose (or avoid) certain specialties, depending on whether they are more eager to produce scientific articles or to enjoy an affluent lifestyle that they consider well-deserved. Registration: osf.io/7d4bk.