User Study of the Effectiveness of a Privacy Policy Summarization Tool

Abstract

The complexity of privacy policies makes it difficult for users to understand its content. In order to solve this, tools exist that analyze and summarize those privacy policies, and present the results in a standardized visual format. The use of these tools can make it possible to analyze any privacy policy, that is, they have the advantage of scale, unlike processes that require manual classification. However, there is scarce research on their effectiveness and how users perceive them. In this paper, an experimental survey was conducted to evaluate whether one such tool, PrivacyGuide, could communicate risk and increase interest in the content of the privacy policy itself. The survey was conducted in Japan with Japanese participants, and considered two languages of the privacy policy, Japanese and English. The results show that interest in the privacy policy increased after viewing the privacy policy summary. On the other hand, risk communication was limited to the case of an English language privacy policy. In addition, survey participants also provided positive and negative feedback about the tool: there was interest in using the tool in a variety of scenarios, but there was also lack of trust in the results. The findings suggest that privacy policy summarization tools have potential to help users, but that there are barriers for adoption of the tool.