Modular Survey Design: Experimental Manipulation of Survey Length and Monetary Incentive Structure

Abstract

Survey length and monetary incentives are design features that have been found to affect survey participation in opposing ways. Declining response rates make these key design features even more important. However, survey length has received surprisingly little attention, possibly being viewed as a static survey attribute rather than an alterable design feature. In addition, experimental tests of the interaction between survey length and monetary incentives could not be found, despite calls for the need for such research and theories that support such an interaction. Leverage-salience theory leads to opposing expectations, depending on the relative importance people give to different factors affecting survey participation. Making the survey request in parts, to reduce burden, may increase participation. Yet, it may instead decrease participation relative to the status quo of offering the full incentive for the entire survey if incentives play a larger role than survey length. When factoring both burden and motivation, the results for survey length could likely depend on the structure of monetary incentives. We designed an experiment to test a modularized survey, ending one part of the survey with a request to complete another part. Respondents in the modular survey design were also assigned to proportionate or disproportionate incentives-offering a larger proportion of the incentive for the initial survey request. Surprisingly, the two-part survey request led to not lower but greater nonresponse. There was increased potential for nonresponse bias attributable to nonresponse to the second request, without reduction in item nonresponse and measurement error. Disproportionate allocation of incentives across modules seemed to increase overall participation compared with proportionate allocation. The lack of a beneficial effect of a request for a shorter survey and the results on incentive allocation have important implications for survey practitioners. Furthermore, the inconsistency in findings on survey length requires further research to identify mediating factors.