Using a Low-Cost Electroencephalograph for Task Classification in HCI Research Johnny Chung Lee Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213 johnny@cs.cmu.edu Desney S. Tan Microsoft Research One Microsoft Way, Redmond, WA 98052 desney@microsoft.com ABSTRACT Modern brain sensing technologies provide a variety of methods for detecting specific forms of brain activity. In this paper, we present an initial step in exploring how these technologies may be used to perform task classification and applied in a relevant manner to HCI research. We describe two experiments showing successful classification between tasks using a low-cost off-the-shelf electroencephalograph (EEG) system. In the first study, we achieved a mean classi- fication accuracy of 84.0% in subjects performing one of three cognitive tasks - rest, mental arithmetic, and mental rotation - while sitting in a controlled posture. In the second study, conducted in more ecologically valid setting for HCI research, we attained a mean classification accuracy of 92.4% using three tasks that included non-cognitive fea- tures: a relaxation task, playing a PC based game without opponents, and engaging opponents within the game. Throughout the paper, we provide lessons learned and dis- cuss how HCI researchers may utilize these technologies in their work. Categories and Subject Descriptors: H.1.2 [User/Machine Sys- tems] H.5.2 [User Interfaces]: Input devices and strategies B.4.2 [Input/Output Devices]: Channels and controllers J.3 [Life and Medical Sciences]. General Terms: Human Factors, Experimentation. Keywords: Brain-Computer Interface, human cognition, physical artifacts, task classification, Electroencephalogram (EEG). INTRODUCTION For generations, humans have fantasized about the ability to communicate and interact with machines through thought alone or to create devices that can peer into a person���s thoughts. These ideas have captured the imagination of hu- mankind in the form of ancient myths and modern science fiction stories. However, only in recent decades have ad- vances in neuroscience and brain sensing technologies made measurable progress toward achieving that vision. These technologies allow us to monitor the physical proc- esses within the brain that correspond with certain forms of thought. Primarily driven by growing societal recognition for the needs of people with physical disabilities, researchers have used these technologies to build brain-computer interfaces (BCIs), communication systems that do not depend on the brain���s normal output pathways of peripheral nerves and muscles [17]. A conceptual illustration of a BCI system is shown in Figure 1. In these systems, users explicitly ma- nipulate their brain activity instead of using motor move- ments to produce signals that can be used to control com- puters or communication devices. The impact of this work is extremely high, especially to those who suffer from dev- astating neurodegenerative diseases such as amyotrophic lateral sclerosis, which eventually strips an individual of all voluntary muscular activity while leaving cognitive func- tion intact. Although removing the need for motor movements in com- puter interfaces is challenging and rewarding, we believe that the full potential of brain sensing technologies as an input mechanism lies in the extremely rich information it could provide about the state of the user. Having access to this state is valuable to HCI researchers because it may al- low us to derive more direct measures of traditionally elu- Figure 1 ��� A conceptual illustration of a Brain-Computer Inter- face using EEG signals for task classification. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. UIST���06, October 15���18, 2006, Montreux, Switzerland. Copyright 2006 ACM 1-59593-313-1/06/0010...$5.00. 81

sive phenomena such as task engagement, cognitive work- load, surprise, satisfaction, or frustration. These measures could open new avenues for evaluating systems and inter- faces. Additionally, knowing the state of the user as well as the tasks they are performing may provide key information that would allow us to design context sensitive systems that adapt themselves to optimally support the state of the user. The work we present in this paper is an initial step in ex- ploring how BCI technology can be applied to HCI re- search. First, we demonstrate that effective exploration in this field can be accomplished using low-cost sensing equipment and without extensive medical expertise. An experiment we conducted shows that we were able to attain 84.0% mean accuracy classifying three different cognitive tasks using an off-the-shelf electroencephalograph (EEG) costing only USD$1500. Within this experiment, we pre- sent a reusable experimental design adapted from previous BCI work and discuss lessons learned so that other HCI researchers can build upon our experiences to perform their own explorations. Second, we present a novel approach to performing task classification by utilizing both cognitive and non-cognitive artifacts measured by our EEG as fea- tures for our classification algorithm. In a second experi- ment, we attained a mean classification accuracy of 92.4% on three tasks within a more ecologically valid setting, de- termining various user states while playing a PC based game. We close with a discussion of how this approach can be useful in certain areas of HCI research. BACKGROUND AND RELATED WORK Brain Sensing and EEG Primer The human brain is a dense network consisting of approxi- mately 100 billion nerves cells called neurons. Each neuron communicates with thousands of others to regulate physical processes and produce thought. Neurons communicate ei- ther by sending electrical signals to other neurons through physical connections or by exchanging chemicals called neurotransmitters. Advances in brain sensing technologies enable us to observe the electrical, chemical, or blood flow changes as the brain processes information or responds to various stimuli. In this paper, we focus on the Electroencephalograph (EEG), a technology used everyday in hospitals and clinics and the most commonly used technology in contemporary BCI research. For general reviews of BCI research, see [4,16,25]. Figure 2 provides a table of alternative brain sensing and imaging technologies and their primary disad- vantages for BCI work, especially within the HCI commu- nity [20]. EEG uses electrodes placed on the scalp to measure the weak (5-100��V) electrical potentials generated by brain activity. Each electrode typically consists of a wire leading to a gold-plated disk that is attached to the scalp using con- ductive paste or gel. An EEG records the voltage at each of these electrodes relative to a reference point, which is often simply another electrode on the scalp [7]. Because EEG is a passive measuring device, it is safe for extended and re- peated use, a characteristic crucial for adoption in HCI re- search. The signal provided by an EEG is at best a crude represen- tation of brain activity due to the nature of the detector. Scalp electrodes are only sensitive to macroscopic and co- ordinated firing of large groups of neurons near the surface of brain, and then only when they are directed along a per- pendicular vector relative to the scalp. Additionally, be- cause of the fluid, bone, and skin that separate the elec- trodes from the actual electrical activity, the already small signals are scattered and attenuated before reaching the electrodes. Each input channel of an EEG includes a multi- stage amplifier with a typical gain of 20,000. Unfortunately, this high electrical sensitivity also makes an EEG susceptible to interference from a variety of sources such as physical movement of the person���s body, indoor power lines, and other electronic equipment. BCI research- ers have invested a great deal of effort in creating experi- mental designs, specialized testing facilities and equipment, and software filtering techniques to minimize the presence of these non-cognitive artifacts [5]. However, such a high degree of environmental and experimental control can be impractical for HCI research that aims to eventually func- tion in a typical home or office scenario. In the work pre- sented in this paper, we limited ourselves to a typical office computing environment without any specialized acoustic or electromagnetic insulation. These studies were run in an unmodified office of an active researcher containing multi- ple computers, fluorescent lights, and other typical sources of signal interference found in an office building. Because much of the work in BCI has grown out of the rehabilitation engineering and neuroscience domains, a large portion of previous research has used high-end de- vices costing between USD$20,000-250,000 [e.g. see sys- tems from www.biosemi.com or www.egi.com]. We were unable to find previous examples of successful BCI re- Brain Sensing Technology Primary Disadvantage Electrocorticogram (ECoG) Highly invasive, surgery Magneto-encephalography (MEG) Extremely expensive Computed Tomography (CT) Only anatomical data Single Photon Emission Computer- ized Tomography (SPECT) Radiation exposure Positron Emission Tomography (PET) Radiation exposure Magnetic Resonance Imaging (MRI) Only anatomical data Functional Magnetic Resonance Imag- ing (fMRI) Extremely expensive Event-Related Optical Signal / Func- tional Near-Infrared (EROS/fNIR) Still in infancy, cur- rently expensive Figure 2. A table of current brain sensing technologies and their primary disadvantages for HCI research. 82