Assessing content validity: challenges of conducting systematic reviews of patient-reported outcome measures and recommendations to improve the application of COSMIN guidance

Abstract

Purpose: Systematic reviews of Patient-Reported Outcome Measures (PROMs) are essential for selecting appropriate measures for research and clinical practice. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines provide a standardised framework for evaluating measurement properties of PROMs, with content validity considered to be the most important property, yet the most complex and poorly reported. This study aimed to report common challenges researchers face when evaluating content validity in systematic reviews on the quality of PROMs, and to provide clear, experience-informed recommendations to overcome them. Method: A narrative synthesis of challenges in evaluating content validity in systematic reviews of PROMs is presented. Challenges were extracted from Elsman et al.’s umbrella review and from stakeholder consultations with the authors of this manuscript, providing practical recommendations and illustrative examples. Results: Seven challenges were identified: (1) feasibility of conducting a systematic review on the quality of PROMs, (2) identifying all articles describing PROM development for the target population, (3) managing different versions of PROMs, (4) poor reporting in primary studies (5) ambiguity in classifying studies as PROM development or content validity studies, (6) applying the criteria for content validity, specifically, to the results of development studies, and (7) accurately framing the scope of the PROM and the review. Recommendations include narrowing the scope of the review, conducting supplementary searches, treating PROM versions distinctly, and guidance on how to interpret poorly reported studies. Conclusion: These insights support consistent application of COSMIN guidelines and improve PROM selection and use, complementing existing COSMIN materials.