Recent Advances in Questionnaire Design for Program Evaluation

Abstract

Abstract: Evaluation uses questionnaires as a central data-gathering technique, yet researchers often appear unaware of recent developments in questionnaire design. This article reviews issues beyond the creation of standardized questions and the basic rules researchers find useful in data collection. These elementary guidelines remain robust for much evaluation research and should not be abandoned hastily. However, rapid change in the theory underlying questionnaire design has important implications for evaluation. Three themes illustrate these changes. First, magnitude scales and their use in client satisfaction scales show how response categories can improve individual questions. Second, decision theory sees respondents as selecting “correct” responses from a portfolio of potential answers. In this view, the answer to a question is conditioned by the values held by the respondent and his or her perception about the risks of revealing true feelings. Third, in some cases it is possible to cast the entire questionnaire into a framework that replicates how choices are made by respondents. In one application, the questionnaire simulates the decision-making of a consumer in the market place. Each of these three themes is applied to problems evaluators face in data collection involving surveys and questionnaires.