UX-Log: Understanding Website Usability through Recreating Users' Experiences in Logfiles

Abstract

Web server log files are an inexpensive, automatically captured recording of a user experience (UX) on a website. In this paper, a tool, UX-Log, used logfiles to recreate users' experiences with the purpose of gaining usability insights. To evaluate the effectiveness and value of this, 10 usability experts watched recreated user experiences using UX-Log and were then asked to infer users' goals, strategies, successes or failures, and proficiencies; and afterwards, rate UX-Log across multiple dimensions. Both user experience recreation and UX-Log proved successful for gaining usability insights; usability experts were able to infer users' goals, strategies, successes or failures, and proficiencies. They were able to do this without training, without familiarity of the website, and without domain knowledge of the subject depicted in the user experiences. However, they were only able to infer users' overarching goal, not specific goal criteria, and were only able to determine relative proficiencies after viewing both user experiences. They also expended a good deal of mental effort trying to comprehend ambiguous user interactions, and identified needed improvement in the UX-Log interface.