Research Article The Wick in the Candle of Learning Epistemic Curiosity Activates Reward Circuitry and Enhances Memory Min Jeong Kang,1 Ming Hsu,2 Ian M. Krajbich,1 George Loewenstein,3 Samuel M. McClure,4 Joseph Tao-yi Wang,5 and Colin F. Camerer1 1Division of Humanities and Social Sciences, California Institute of Technology, Pasadena 2Beckman Institute for Advanced Science and Technology and Department of Economics, University of Illinois at Urbana-Champaign 3Department of Social and Decision Sciences, Carnegie Mellon University 4Department of Psychology, Stanford University and 5Department of Economics, National Taiwan University ABSTRACT���Curiosity has been described as a desire for learning and knowledge, but its underlying mechanisms are not well understood. We scanned subjects with func- tional magnetic resonance imaging while they read trivia questions. The level of curiosity when reading questions was correlated with activity in caudate regions previously suggested to be involved in anticipated reward. This finding led to a behavioral study, which showed that sub- jects spent more scarce resources (either limited tokens or waiting time) to find out answers when they were more curious. The functional imaging also showed that curiosity increased activity in memory areas when subjects guessed incorrectly, which suggests that curiosity may enhance memory for surprising new information. This prediction about memory enhancement was confirmed in a behavioral study: Higher curiosity in an initial session was correlated with better recall of surprising answers 1 to 2 weeks later. Curiosity is the complex feeling and cognition accompanying the desire to learn what is unknown. Curiosity can be both helpful and dangerous. It plays a critical role in motivating learning and discovery, especially by creative professionals, increasing the world���s store of knowledge. Einstein, for example, once said, ������I have no special talents. I am only passionately curious������ (Hoffmann, 1972, p. 7). The dangerous side of curiosity is its association with exploratory behaviors with harmful con- sequences. An ancient example is the mythical Pandora, who opened a box that unleashed misfortunes on the world. In modern times, technology such as the Internet augments both good and bad effects of curiosity, by putting both enormous amounts of information and potentially dangerous social en- counters a mouse-click away. Despite the importance of human curiosity, its psychological and neural underpinnings remain poorly understood. Philoso- phers and psychologists have described curiosity as an appetite for knowledge, a drive like hunger and thirst (Loewenstein, 1994), the hunger pang of an ������info-vore������ (Biederman & Vessel, 2006, p. 247), and ������the wick in the candle of learning������ (William Arthur Ward, cited by Wikiquote, 2008). In reinforcement learning, a novelty bonus is used to motivate the choice of un- explored strategies (Kakade & Dayan, 2002). Curiosity can be thought of as the psychological manifestation of such a novelty bonus. A theory guiding our research holds that curiosity arises from an incongruity, or information gap���a discrepancy between what one knows and what one wants to know (Loewenstein, 1994). This theory assumes that the aspired-to level of knowl- edge increases sharply with a small increase in knowledge, so that the information gap grows with initial learning. When one is sufficiently knowledgeable, however, the gap shrinks, and cu- riosity falls. If curiosity is like a hunger for knowledge, then a small ������priming dose������ of information increases the hunger, and the decrease in curiosity from knowing a lot is like being satiated by information. Address correspondence to Colin F. Camerer, Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, CA 91125, e-mail: camerer@hss.caltech.edu. PSYCHOLOGICAL SCIENCE Volume 20���Number 8 963 Copyright r 2009 Association for Psychological Science

In the information-gap theory, the object of curiosity is an unconditioned rewarding stimulus: unknown information that is anticipated to be rewarding. Humans (and other species, such as cats and monkeys) will expend resources to find out information they are curious about, much as rats will work for a food reward (Loewenstein, 1994). On the basis of this observation, we hy- pothesized that the striatum would be linked to curiosity be- cause a growing body of evidence suggests that activity in the human striatum is correlated with the level of reward signals (Hare, O���Doherty, Camerer, Schultz, & Rangel, 2008 Knutson, Westdorp, Kaiser, & Hommer, 2000 McClure, York, & Mon- tague, 2004 O���Doherty, 2004). Guided by these ideas, we explored the neural correlates of curiosity in one study and tested the hypotheses derived from its findings in two additional studies. In all studies, subjects were presented with a series of trivia questions chosen to create a mixture of high and low epistemic curiosity1 (Fig. 1a). Subjects read each question, guessed the answer, rated their curiosity about the question, and rated how confident they were that they knew the answer (P). Then the question was presented again, followed by the correct answer (Fig. 1b). In the first experiment, subjects read the questions during functional magnetic resonance imaging (fMRI). In the second experiment, they performed the same task without scanning, and their memory for answers was tested in a follow-up session 1 to 2 weeks later. In the third experiment, we behaviorally tested whether curiosity is indeed a form of reward anticipation. EXPERIMENT 1 Method Participants and Task Nineteen Caltech students were scanned (average age 5 21.7 3.5 years 14 males, 5 females 18 right-handed, 1 left-handed). They earned $20 for participation. Informed consent was ob- tained using a consent form approved by the internal review board at Caltech. The stimuli used in the task were 40 trivia questions on various topics (these and other materials can be viewed in Experimental Materials in the Supporting Information available on-line see p. 973). They were designed to measure curiosity about semantic knowledge, and pretesting indicated that they evoked a range of curiosity levels (for sample ques- tions, see Fig. 1a). Participants were instructed that after reading each question, they should silently guess the answer and indicate their curiosity about the correct answer and their confidence in their guess. Then the question was presented again, followed by the correct answer (for a time line, see Fig. 1b). Curiosity ratings were on a scale from 1 to 7, and for all the analyses that follow, the raw curiosity ratings were individually normalized (i.e., the individual���s mean curiosity was subtracted from each rating, and the resulting value was divided by that individual���s standard deviation). The confidence scale ranged from 0 to 100%, but was rescaled to range from 0 to 1. Verbal or typed responses are not easy to collect in a scanner, so subjects provided their initial guesses outside of the scanner upon completion of the task. fMRI Acquisition and Analysis Data were acquired using a 3-T Siemens (Erlangen, Germany) Trio scanner at Caltech. A set of high-resolution (0.5 0.5 1.0 mm3) T1-weighted anatomical images was first acquired to en- able localization of functional images. Whole-brain T2n- weighted echo-planar images with blood-oxygenation-level- dependent (BOLD) contrast were acquired in 32 axial slices (64 64 voxels 3-mm thickness and 3-mm in-plane resolution) at a repetition time of 2,000 ms and echo time of 30 ms. The scan sequences were axial slices approximately parallel to the anterior commissure���posterior commissure axis. The fMRI data were preprocessed using SPM2 (Wellcome Depart- ment of Imaging Neuroscience, Institute of Neurology, London, United Kingdom). Functional scans were first corrected for slice timing via linear interpolation. Motion correction of images was performed using a six-parameter affine transformation followed by nonlinear warping using basis functions (Ashburner & Fris- ton, 1999). Finally, images were smoothed with a Gaussian kernel of 8-mm full-width at half-maximum. The data analysis was conducted using the random-effects general linear model (GLM) for event-related designs in SPM2. Curiosity Median-Split Analysis. Each subject���s trials were split into two conditions (high or low) according to where they fell relative to that individual���s median curiosity level. Then all five epochs in each trial (first presentation, curiosity rating, confi- dence rating, second presentation, and answer display) were classified as being in the high- or low-curiosity condition ac- cording to the condition to which the whole trial had been as- signed. Thus, there were two curiosity conditions for each epoch, resulting in a total of 10 separate regressors of interest. Each regressor was time-locked to stimulus presentation. A GLM including these 10 regressors plus regressors of no interest was estimated. The 10 regressors of interest were modeled using box- car functions with the length of each epoch (e.g., the presenta- tion time for the first answer) as the corresponding box-car du- ration. We then calculated contrasts to compare the effects of high versus low curiosity. Curiosity Modulator Analysis. We also examined whether the brain activations identified in the median-split analysis in- creased linearly with curiosity level, rather than being associ- ated with two levels (high or low) of curiosity. We estimated a GLM in which normalized curiosity was a parametric modulator for each of the five epochs. 1 Epistemic curiosity refers to a desire to acquire knowledge and applies mainly to humans (Loewenstein, 1994). 964 Volume 20���Number 8 Curiosity, Reward, and Memory

First Presentation (12���15 s) Fixation (4���6 s) Fixation (4���6 s) Fixation (4���6 s) Curiosity Rating (Self-Paced) Confidence Rating (Self-Paced) Second Presentation (3���5 s) Answer Display (4���6 s) Time Confidence Level (Jittered) Normalized Curiosity What instrument was invented to sound like a human singing? What instrument was invented to sound like a human singing? What instrument was invented to sound like a human singing? What instrument was invented to sound like a human singing? Violin a b c Violin What is the name of the galaxy that Earth is a part of? Milky Way + + + + How curious are you? Not curious at all Very curious How confident are you (%)? or tip of tongue (TOT)? 0 10 20 30 40 50 60 70 80 90 100 TOT Trial Onset Fixation (4���6 s) 3 2 1 0 ���1 ���2 ���3 0 0.2 0.4 0.6 0.8 1 1 2 3 4 5 6 7 Fig. 1. Experimental protocol and behavioral results in Experiment 1: (a) sample questions, (b) trial sequence, and (c) distribution of curiosity ratings as a function of confidence. The questions in (a) are examples of items with relatively high (left average score 5 5.72) and low (right average score 5 2.28) curiosity ratings. For the scatter plot (c), all confidence ratings were jittered by adding random numbers U [ 0.01, 0.01], to convey data density. There was also a ������tip of the tongue������ response option, but there were too few of these responses to analyze, so they were excluded. The large, unfilled circles indicate mean curiosity at each confidence level. Diamonds indicate individual observations. The solid curve is the regression line of curiosity against confidence, P, and P(1 P). The estimated regression was calculated as follows: curiosity 5 0.49 ��� 0.39P 1 4.77P(1 P) 1 residual curiosity. Volume 20���Number 8 965 M.J. Kang et al.