Using multiple research methods to inform transformation of a software application into a web survey for businesses

Abstract

Every 5 years the U.S. Census Bureau collects detailed economic data from over 4 million business establishments for an economic census. Starting in 1997, a software application was offered as an alternative to paper forms. In 2018, businesses will be answering this complex survey using the Web as their only reporting option. In order to prepare for this change in platform, the Census Bureau undertook a multi-year/multi-method research program to identify key requirements and to test prototypes and early versions prior to launching the Web survey. In addition to ensuring that key functionality from the legacy software was transferred to the Web instrument, researchers also took the opportunity to identify potential enhancements to the design based on feedback obtained from respondents, internal Census Bureau staff, and user experience experts. Various research methods were used to guide this transformation, using an iterative approach that took advantage of unique contributions offered by each technique. The process began with a detailed requirements gathering interviews with respondents and other key stakeholders and experts. A task analysis was conducted investigating processes and activities used by business respondents to complete surveys. Debriefing interviews were conducted with business respondents to evaluate features of the legacy software application, determining functionality critical for aiding respondents’ work while also detecting tools that were under-utilized and identifying obstacles to their use. Usability testing was conducted using low-fidelity prototypes and early semi-functioning versions of the Web survey in order to test new and redesigned elements. Along the way, paradata from the legacy software and implementation of an early version of the Web survey were analyzed to further evaluate existing and revised features. This poster will display how these various research methods complemented and augmented one another. It will also highlight challenges and lessons learned.