Counting methods introduced into the bibliometric research literature 1970–2018: A review

Abstract

This review investigates (a) the number of unique counting methods, (b) to what extent counting methods can be categorized according to selected characteristics, (c) methods and elements to assess the internal validity of counting methods, and (d) to what extent and with which characteristics counting methods are used in research evaluations. The review identifies 32 counting methods introduced from 1981 to 2018. Two frameworks categorize these counting methods. Framework 1 describes selected mathematical properties and Framework 2 describes arguments for choosing a counting method. Twenty of the 32 counting methods are rank dependent, fractionalized, and introduced to measure contribution, participation, etc. of an object of study. Next, three criteria for internal validity are used to identify five methods that test the adequacy, two elements that test the sensitivity, and three elements that test the homogeneity of counting methods. Finally, a literature search finds that only three of the 32 counting methods are used by four research evaluations or more. Two counting methods are used with the same characteristics as defined in the studies that introduced the counting methods. The review provides a detailed foundation for working with counting methods, and many of the findings provide bases for future investigations of counting methods.