The 100,000 participant laboratory - A crowd-centered approach to design and evaluate the usability of mobile apps

Abstract

Mobile apps play important roles in our daily lives. Software developers typically follow the user-centered design (UCD) approach to develop products that optimize usability. However, most usability methodologies require lab experiments or field studies. Usability experiments require higher costs and efforts to prepare related tasks, such as selecting the environmental setting, subject recruiting and conducting the study. Compared to the laboratory experiment of personal computers (PC), mobile usability laboratory settings present greater challenges related to controlling the research context variables. The purpose of this study is to build an online usability lab by applying Internet and crowdsourcing technologies. The lab allows researchers to invite Internet users to participate in the usability experiment online. This study follows the usability engineering theoretical framework to define mobile usability contextual factors. The clouded-based online usability lab was introduced as a conceptual framework to support both Android and iOS mobile apps. Mobile app developers can upload their apps to the lab system to collect and monitor usability data and track the behavioral data of human-computer interaction. The ultimate goal of this lab was to help researchers conduct usability experiments in real world contexts and develop the dynamic mental model of product users.