Does the Number of Response Categories Impact Validity Evidence in Self-Report Measures? A Scoping Review

Abstract

The optimal number of responses categories (NRC) is among the most discussed, yet least decided, topic in self-report measures. In addition, there is a dearth of scoping reviews that summarize its impact on validity evidence (e.g., evidence-based on internal structure). To that end, we conducted a scoping review of methodological literature to provide self-report measures developers and applied researchers with evidence-based recommendations when selecting the optimal NRC. Given the inconsistent results reported in previous research, a key recommendation, when conducting cognitive interviews, is to investigate the interpretation of response options by a sample of potential participants who have varying perspectives. This procedure is paramount to ascertain that response options are accurately interpreted and function as intended. The present scoping review is expected to become a valuable resource for applied researchers and practitioners to make informed decisions about the optimal NRC, taking into account validity evidence, and therefore contributes to the literature of educational and psychological measurement and research methods.