Addressing users' privacy concerns for improving personalization quality: Towards an integration of user studies and algorithm evaluation

Abstract

Numerous studies have demonstrated the effectiveness of personalization using quality criteria both from machine learning/data mining and from user studies. However, a site requires more than a high-performance personalization algorithm: it needs to convince its users to input the data needed by the algorithm. Today's Web users are becoming increasingly privacy-conscious and less willing to disclose personal data. How can the advantages of personalization (and hence, of disclosure) be communicated effectively, and how can the success of such strategies be measured in terms of improved personalization quality? In this paper, we argue for a tighter integration of the HCI and computational issues involved in these questions. We first outline the problems for personalization that arise from the combination of users' privacy concerns and sites' current policies of dealing with privacy issues. We then describe the results of an experiment that investigated the effects of changes to a site's interface on users' willingness to disclose data for personalization. This is followed by an overview of studies of the sensitivity of mining algorithms to changes in the availability of these types of data. Based on this, we outline a research agenda for future evaluation studies and user agent design. © Springer-Verlag Berlin Heidelberg 2005.