Construction of an evaluation index system for determining the academic impact of military medical scholars

Abstract

Introduction: Academic papers are an essential manner for describing new ideas and consolidating existing concepts in the field of military medicine. The academic impact of military medical publications reflects the extent and depth of recognition, acceptance and utilisation of the concepts transmitted in these publications. The aim of this research was to construct an evaluation index system suitable for evaluating the academic influence of scholars in the field of military medicine. Methods: Using the Delphi consensus methodology, 30 experts from the field of military medicine, military medical information and library and information science were asked during three rounds of questioning to score the feasibility and importance of indicators that could be used to determine academic impact. An analytic hierarchy process method was used to calculate the relative weighting of each indicator in determining the final level of academic impact. Results: Eight evaluation indicators were agreed on to potentially determine academic impact. These comprised: 'Web of Science documents', 'Citation impact', 'h-index', 'Percentage of international collaborations', 'Percentage of the top 10% of the cited frequency', 'Category normalised citation impact', 'Percentage of documents cited' and 'The number of F1000 Recommended papers'. Conclusions: The evaluation index system determined from this study combines the advantages of both qualitative and quantitative recognised evaluation indicators, which are subsequently weighted according to their importance in the field of military medicine. It is hoped that this framework will provide a manner in the future for comparing the potential academic impact of military medical scholars.