Timing estimates for complex programmed surveys

Abstract

Developing a method for estimating average survey completion time—easily, accurately, and repeatedly—was a many-years-long aspiration for the International Tobacco Control Policy Evaluation (ITC) Project. This article describes the three survey length estimation methods used for the ITC Project’s complex programmed surveys, with their respective merits and disadvantages. Method I, Read-Throughs, required staff to read through different versions of the survey. Automating this approach in Method II, Word Counts, replaced staff reading with question word counts and a postulated reading speed, but still required creation of multiple versions that were timed separately. Method III, Weighted Questions, also automated, was streamlined by weighting individual questions, rather than survey versions, by the proportion of respondents likely to answer them. The two automated methods greatly reduced staff time cost, provided surprisingly accurate time estimates, and made it possible to quickly re- estimate the length after each survey revision. Other survey researchers using surveys with complex branching may benefit from our experimentation and result