Evaluating user experience under location quality variations: A framework for in-the-wild studies

Abstract

While lab-based approaches to evaluation of location-based services (LBSs) allow faster, cost-effective and less complicated evaluation of many aspects, they are limited in evaluating certain aspects, especially the ones related to usability and user experience (UX). Variations in the location information quality can negatively affect location-based UX. Lab-based evaluations are limited in factoring users' actual context, which includes location quality variations into the evaluation process. Therefore, to evaluate actual location-based UX, we need to evaluate LBSs in-the-wild with users under variations in the quality of location information. Currently, there is a lack of standard tools, methods, and frameworks for this purpose. Motivated by this factor, in this paper, we describe the Location Uncertainty Injection Framework (LUIF) for evaluating mobile LBSs in-the-wild with users under location quality variations. We also present the results of a study conducted to gain initial feedback on the potential use cases of the framework.