P. Dourish and A. Friday (Eds.): Ubicomp 2006, LNCS 4206, pp. 177

–

193, 2006.
' Springer-Verlag Berlin Heidelberg 2006
SenseCam: A Retrospective Memory Aid
Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi,
James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur
*
, and Ken Wood
Microsoft Research, 7 JJ Thomson Avenue,
Cambridge, CB3 0FB, UK
{shodges, lyn, v-emmabe, shahrami, i-james, v-alexbu,
gavin.smyth, krw}@microsoft.com
http://research.microsoft.com/sendev/
* Cambridge Memory Clinic, R3 Neurosciences, Box 83,
Addenbrooke’s Hospital, Cambridge, CB2 2QQ, UK
narinder.kapur@addenbrookes.nhs.uk
Abstract. This paper presents a novel ubiquitous computing device, the Sense-
Cam, a sensor augmented wearable stills camera. SenseCam is designed to cap-
ture a digital record of the wearer’s day, by recording a series of images and
capturing a log of sensor data. We believe that reviewing this information will
help the wearer recollect aspects of earlier experiences that have subsequently
been forgotten, and thereby form a powerful retrospective memory aid. In this
paper we review existing work on memory aids and conclude that there is scope
for an improved device. We then report on the design of SenseCam in some de-
tail for the first time. We explain the details of a first in-depth user study of this
device, a 12-month clinical trial with a patient suffering from amnesia. The re-
sults of this initial evaluation are extremely promising; periodic review of im-
ages of events recorded by SenseCam results in significant recall of those
events by the patient, which was previously impossible. We end the paper with
a discussion of future work, including the application of SenseCam to a wider
audience, such as those with neurodegenerative conditions such as Alzheimer’s
disease.
1 Introduction
Human memory is all too fallible – most of us frequently forget things that we have to
do, and often find it hard to recall the details around what we have previously done.
Of course, for those with clinically diagnosed memory disorders – which are by their
nature more severe than those found in the average population – these issues are par-
ticularly troublesome. One example of such a diagnosis is acquired brain injury,
which occurs either through a disease with lasting effect on brain tissue, or a trau-
matic incident like a car accident. Another example, perhaps of more significance in
an aging population, is neurodegenerative disease – essentially an illness which dam-
ages the brain such that there is no possibility of recovery. The most prevalent neu-
rodegenerative disease is Alzheimer’s disease.
The effects of acquired brain injuries, neurodegenerative diseases and aging in gen-
eral vary greatly from patient to patient. In a relatively moderate form, there may be

178 S. Hodges et al.
little noticeable effect – perhaps a patient will be frustrated from time-to-time at their
inability to organize themselves as well as they used to. At a more extreme level, a
patient may suffer from a near complete inability to remember. Patients with moder-
ate to severe memory problems may fail to remember future intentions, such as buy-
ing milk, or making and keeping appointments (prospective memory). Prospective
memory problems clearly have a big impact on the ability of a patient to look after
themselves on a day-to-day basis. A failure of past (retrospective) memory, however,
and in particular episodic or autobiographical memory (i.e. the memory of things the
patient has done, as opposed to their semantic memory for factual information) is
actually critical for a patient to enjoy any real quality of life. This is firstly because
nearly all future actions are based on past experiences, so practical day-to-day plan-
ning is very difficult when autobiographical memory is impaired. Secondly, and per-
haps more importantly, a memory of past experiences is critical to a patient’s ‘self-
concept’. For example, without a memory of shared experiences it is very hard to
maintain any kind of relationship, whether it is professional, social or personal. This
in turn frequently affects the patient’s self-esteem, which can have significant knock-
on effects on their well-being [13].
This paper presents a novel ubiquitous computing device, a wearable camera that
keeps a digital record of the events that a person experiences. The nature of the device
is to take these recordings automatically, without any user intervention and therefore
without any conscious effort. We call the device SenseCam [4, 7] because two of the
main components of its operation are sensing its environment and using a built-in
stills camera to record images.
1
The rationale behind SenseCam is that having cap-
tured a digital record of an event, it can subsequently be reviewed by the wearer in
order to stimulate their memory.
This paper describes the technology behind SenseCam in detail for the first time,
presenting the design of our latest prototype (version 2.3) and discussing some of the
issues that have come to light during its development. It also outlines a desktop appli-
cation that has been developed in tandem with the hardware, to facilitate the viewing
of data captured by SenseCam. The prototype hardware and software have been tested
in an early clinical trial by a patient suffering from amnesia, and the promising results
of this work are presented and discussed.
2 Related Work
The use of external memory aids to help people to compensate for their memory defi-
cits is thought to be one of the most valuable and effective ways to aid rehabilitation
(see [13] for a review). Most external memory aids serve to improve prospective
memory; that is, they help people to remember to keep appointments, take medication
and so forth. Many different types of device are available, including calendars, dia-
ries, alarm watches, whiteboards, timers, post-it notes and more sophisticated tools,
such as hand-held electronic schedulers. The combined use of these prospective aids
has been effective in increasing independence in brain injured patients [13].

1
“Cam” is also significant because the camera was conceived and built in Cambridge, UK.