Evaluating patient-based outcome measures for use in clinical trials

Abstract

Background: 'Patient-based outcome measure' is a short-hand term referring to the array of questionnaires, interview schedules and other related methods of assessing health, illness and benefits of health care interventions from the patient's perspective. Patient-based outcome measures, addressing constructs such as health-related quality of life, subjective health status, functional status, are increasingly used as primary or secondary end-points in clinical trials. Objectives: To describe the diversity and reasons for diversity of available patient-based outcome measures. To make clear that criteria investigators should have in mind when they select patient-based outcome measures for use in a clinical trial. Methods: Data sources: Literature was identified by a combination of electronic searches of databases, handsearching of selected journals and retrieval of references cited in available literature. Databases used included MEDLINE, EMBASE, CINAHL, PsychLIT and Sociofile. Study selection: A set of explicit criteria were used for selection of literature. Articles were included if they focused on any methodological aspect of patient-based outcome measures (for example, methods of evaluating such measures, psychometric evaluation of measures, comparative studies of measures, studies reporting validation of measures). Studies were excluded if they only reported use of a measure without evaluation, focused only on cross-cultural issues, focused only on clinician-based outcome measures or discussed economic utility theory only without considering measurement. A total of 5621 abstracts and articles were identified by initial searches as potentially relevant. However, after assessment, 391 key references were selected as useful to the objectives of the review. A further 22 references were incorporated into the final version as a result of comments from external experts and referees. Data synthesis: A first draft synthesising the evidence was produced by the first author of this review (RF) and extensively critiqued by the other three authors. A revised version was then submitted for evaluation to a panel of ten experts recruited to represent a wide range of areas of expertise (including clinical medicine, clinical trials, health economics, health services research, social sciences and statistics). Feedback from this panel was read and discussed by the authors of the review and a third version of the review drafted. The final version is a quasi- consensus view from individuals with a wide range of expertise. Results: Diversity of patient-based outcome measures: Seven major types of instrument can be identified in the literature: disease-specific, site-specific, dimension-specific, generic, summary item, individualised, utility. Concepts, definitions and theories of what such instruments measure are generally not clearly or consistently used. For example, there is little consistency of use or agreement as to the meaning of key terms such 'quality of life' and 'health-related quality of life'. The intended purpose and content of types of instruments vary. There are advantages and disadvantages to each of the different type of instrument when used in a particular clinical trial. Criteria for selecting patient-based outcome measures: There are eight criteria that investigators should apply to evaluate candidate patient-based outcome measures for any specific clinical trial: appropriateness, reliability, validity, responsiveness, precision, interpretability, acceptability, feasibility. These criteria are not consistently defined and the literature associated with the criteria cannot be summarised in clear, explicit and unambiguous terms. It is not possible from available evidence to rank order the relative importance of the eight criteria in relation to decisions about selection of measures to include in a trial. Appropriateness requires that investigators consider the match of an instrument to the specific purpose and questions of a trial. Reliability requires that an instrument is reproducible and internally consistent. Validity is involved in judging whether an instrument measures what it purports to measure. Responsiveness in this context addresses whether an instrument is sensitive to changes of importance to patients. Precision is concerned with the number and accuracy of distinctions made by an instrument. Interpretability is concerned with how meaningful are the scores from an instrument. Acceptability addresses how acceptable is an instrument for respondents to complete. Feasibility is concerned with the extent of effort, burden and disruption to staff and clinical care arising from use of an instrument. Conclusions and recommendations: Investigators need to make their choice of patient-based outcome measures for trials in terms of the criteria identified in this review. Developers of instruments need to make evidence available under the same headings. By means of the above criteria, further primary research and consensus-type processes should be used to evaluate leading instruments in the different fields and specialties of health care to improve use of patient-based outcome measures in research. Primary research is needed either in the form of methodological additions to substantive clinical trials (for example comparing the performance of two or more measures) or studies of leading measures with methodology as the primary rationale.