Software or Wetware? Discovering When and Why People Use Digital Prosthetic Memory Vaiva Kalnikait�� Sheffield University 211 Portobello St, Sheffield, S1 4DP, UK. v.kalnikaite@sheffield.ac.uk Steve Whittaker Sheffield University 211 Portobello St, Sheffield, S1 4DP, UK. s.whittaker@sheffield.ac.uk ABSTRACT Our lives are full of memorable and important moments, as well as important items of information. The last few years have seen the proliferation of digital devices intended to support prosthetic memory (PM), to help users recall experiences, conversations and retrieve personal information. We nevertheless have little systematic understanding of when and why people might use such devices, in preference to their own organic memory (OM). Although OM is fallible, it may be more efficient than accessing information from a complex PM device. We report a controlled lab study which investigates when and why people use PM and OM. We found that PM use depended on users��� evaluation of the quality of their OM, as well as PM device properties. In particular, we found that users trade-off Accuracy and Efficiency, preferring rapid access to potentially inaccurate information over laborious access to accurate information. We discuss the implications of these results for future PM design and theory. Rather than replacing OM, future PM designs need to focus on allowing OM and PM to work in synergy. Keywords Memory, Prosthetic Memory, Digital Memory, Speech Browsing, Speech Retrieval, Notes, Remembering. ACM Classification Keywords H5.2 Information interfaces and presentation (e.g., HCI): User Interfaces ��� auditory (non-speech) feedback, evaluation/methodology, user-centered design. INTRODUCTION AND MAIN QUESTIONS Our lives are full of memorable and important moments, as well as important items of information. But human memory is fallible, and paper-based prosthetic memories (PM) such as notebooks, sticky notes, calendars and diaries are facts of life. This paper aims to understand how and when new forms of PM can assist users in remembering information they might otherwise forget. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2007, April 28���May 3, 2007, San Jose, California, USA. Copyright 2007 ACM 978-1-59593-593-9/07/0004...$5.00. In his oft-cited vision of the future, Vannevar Bush [2] proposed Memex, a tool designed to help users remember and index information they have previously encountered. In the last 15 years this vision has become a reality. Reductions in the cost of digital storage and the emergence of sophisticated recording technologies have led to the development of many different PM systems. One type of PM captures personal experience, whether this is by recording conversations [15,16], or by recording both sound and visual experiences [1,3,9,11]. Other PM systems serve as general repositories for personal data [4,5,6]. Finally there are prospective memory systems designed to help users remember future tasks and commitments [12,13]. With some notable exceptions [4,13,14,16], however, the majority of this work has focused on the development of proof of concept systems rather than on how and why they are used. One critical issue that has not been systematically explored is the relationship between PM and natural organic memory (OM). It is obvious that a well-designed PM can help users to access information that they may otherwise have forgotten, but there may be reasons why users sometimes prefer to rely on OM. For example there is little incentive to use PM when one can remember information unaided. There may also be inefficiencies associated with using PM, as retrieving information from OM is extremely rapid. Compare, for example, the ease and Fig 1. ChittyChatty Graphical User Interface

rapidity of accessing a familiar contact name from OM with the effort of retrieving it from a poorly organised email archive. These PM trade-offs are illustrated in an exploratory study [16]. That study investigated PM use for remembering simple conversations. Users had access to 3 types of PM: (1) Pen and Paper notes, (2) a Dictaphone and (3) Filochat - a device that co-indexed recorded speech and handwritten notes. Although both Filochat and the Dictaphone created verbatim records, users did not always exploit such accurate PM devices to retrieve information. Both user confidence in the accuracy of their OM, and the efficiency of the PM device appeared to influence PM usage - although these factors were not systematically varied. For example, users were unlikely to use PM when this was laborious, e.g. extracting information from a Dictaphone recording. Other research also suggests the factors underlying PM use are not well understood. A number of sophisticated systems have been developed to help ���meeting capture���, i.e. user extraction of key information from a meeting record. However field trials indicate that these digital PMs are not always used in practice to retrieve meeting content [7]. Often users prefer to rely on their own memory or on paper based PMs, such as handwritten notes. The current laboratory study therefore sets out to systematically investigate the role of 4 factors on PM usage: (a) PM Accuracy, (b) PM Efficiency, (c) User Confidence in the accuracy of their own memory and (d) time from the event to be recalled (Retention Interval). We had two main research questions. First, what are the objective costs and benefits of using PM compared with OM. Specifically, what are the trade-offs between the potentially greater accuracy of using PM compared to the efficiency costs incurred in using it? Second, we take a detailed look at PM usage patterns, in particular when and why PM might be used in preference to OM. We investigate these questions by comparing retrieval for spoken conversations using 3 PM devices, including Pen and Paper, a Dictaphone and a new generation of Filochat-like device (ChittyChatty). We examine how these different PMs are used for retrieving different types of information, over different periods of time, and how this compares with OM usage. Specific questions we address are: 1. What are the benefits of using PM compared with OM? Are digital memories more accurate than OM, and are they quicker and more efficient to access? 2. How does PM use relate to people���s perceptions of the accuracy of their OM? E.g. if users are confident they can retrieve information unaided, we would not expect them to use PM. 3. Does PM use depend on the characteristics of the PM device, in particular how easy it is to retrieve information from that device? Obviously, we expect greater use of a PM device that makes retrieval straightforward. In contrast, a hard-to-use PM device such as a Dictaphone might force users to fall back on OM, even when they are unsure they can retrieve information unaided. 4. Does PM use depend on the type of information being retrieved? E.g. we might expect PM use to be more prevalent for complex verbatim information which is hard to recall unaided. 5. Finally does PM use change over time? Do users become more reliant on PM for events that are further in the past? As OM degrades we might expect greater use of PM devices. EXPERIMENTAL METHOD We investigated memory using three different types of prosthesis: Dictaphone (DP) Pen and Paper (PP) and a system called ChittyChatty (CC) which we describe in detail below. Our goal was to test how these different PMs helped users remember everyday conversations. We read a series of conversational stories aloud to users, asking them later to retrieve information about those stories. Users were given a PM to help them remember, but they could also choose to rely on unaided OM. Prostheses ChittyChatty Fig 1 illustrates the ChittyChatty interface. CC is similar to other note-taking systems such as [1,15,16]. Like those systems, it supports memory for conversation using temporal co-indexing [16] of handwritten notes and speech. The main representation is a blank page where users create notes and/or other visual cues while recording a conversation. Users follow their normal practice of taking handwritten notes but each pen stroke is temporally co- indexed with the underlying recorded speech. This allows the notes to be used to access the conversation when users want to re-access recorded speech, they click on a specific note, and the system begins to replay the speech that was being recorded at the moment that note was taken. In this way the notes serve as a visual analogue to the underlying speech, allowing straightforward access to a specific part of the speech. This gives the users a more precise way of accessing a specific part of speech without having to listen to the whole audio again. CC runs on any version of Windows Mobile edition on a PDA, making it portable and easy for taking meeting notes. Dictaphone The Dictaphone (DP) we used was a Sony digital voice recorder ICD-P320. It had standard functions such as record, play, pause, stop, rewind, and fast-forward. Audio was recorded and played back in a standard way. Differences in Efficiency and Accuracy between PM and OM Our three different types of PM have different properties. PP notes are a schematic and incomplete record of what was said, whereas DP and CC offer verbatim records. Retrieval Efficiency is also different for these PM devices.