Iterated Learning and the Cultural Ratchet Aaron Beppu (abeppu@berkeley.edu) Thomas L. Griffiths (tom griffiths@berkeley.edu) Department of Psychology and Cognitive Science Program University of California Berkeley, Berkeley, CA 94720 USA Abstract How does the behavior of individuals in a society influence whether knowledge accumulates over generations? We explore this question using a simple model of cultural evolution as a process of ���iterated learning,��� where each agent in a sequence learns and passes on a piece of information. Using both math- ematical analyses involving rational Bayesian agents and lab- oratory experiments with human participants, we vary whether agents observe data from the environment and what kind of information they receive from the previous agent. Our mathe- matical and empirical results both suggest that merely observ- ing the behavior of other learners is not sufficient to produce cumulative cultural evolution, but that knowledge can be accu- mulated over generations when agents are able to communicate the plausibility of different hypotheses. Introduction What we as humans know about the world is largely informed by what we can learn from others. Our beliefs and behaviors are built up not only by our own interactions with the environ- ment around us, but by observations of the behavior of oth- ers, and by receiving explicit statements of belief. Through this ability to build off the knowledge and behaviors of those around us, human societies are able to accumulate informa- tion, the so-called ���ratchet effect��� (Tomasello, Kruger, & Rat- ner, 1993). Large, lasting changes in our beliefs develop by accruing small, incremental changes over many generations. In this paper, we use a simple model to characterize the con- ditions necessary for this accrual, and compare the predic- tions of this model to experimental results. In particular, we suggest that these changes will accumulate if learners have access to the actual beliefs of their teachers, as opposed to merely observations of behavior informed by those beliefs. Much of the previous discussion about cumulative cultural evolution has focused on the relationship between the teacher and the learner. For example, Tomasello (1993) largely claims that the ratchet effect is a product of sophisticated imi- tative learning, emphasizing the advantages of joint attention and a theory of mind. Gergely and Csibra (2005) suggest that ���natural pedagogy��� plays a role, focusing on how human teachers go out of their way to ease the learning process for children, providing highly tailored information and a greatly enriched learning environment. Both of these mechanisms increase the fidelity of cultural transmission, providing the opportunity for the ratchet effect occur. Rather than focusing on the specific actions or abilities of teachers and learners, we consider the kind of informa- tion passed between generations. We model the process of cultural evolution using an ���iterated learning��� framework (Kirby, 2001), in which successive generations are modeled as a sequence of agents in which each agent learns from the previous agent. Iterated learning models have recently been used in simulations exploring the emergence of linguis- tic structures (Kirby, 2001) and in experiments with human participants revealing inductive biases in specific domains (Kalish, Griffiths, & Lewandowsky, 2007). We explore vari- ants of this framework where each agent learns not only from the behavior of the previous generation, but also from ob- serving the world and from theories summarizing the infer- ences of the previous generation. The consequences of each variant for the accumulation of knowledge are determined through a combination of mathematical analyses of sequences of Bayesian agents and an experiment in which cultural evo- lution is simulated in the laboratory with human participants. Our mathematical and empirical analyses produce two re- sults that help to identify the conditions under which cumu- lative cultural evolution can occur. First, we show that if the information passed from teacher to learner is limited to the learner���s observations of the teacher���s behavior, then cumula- tive cultural evolution will not take place. We then go on to show that cumulative cultural evolution can occur if teachers give learners information about which hypotheses they be- lieve are plausible, rather than just behaviors consistent with the most plausible hypothesis. These results suggest that a capacity for communicating such beliefs may be necessary in order for the cultural ratchet to operate. The plan of the paper is as follows. The next section intro- duces our formal framework, and summarizes our mathemati- cal results. We then outline the motivation for our experiment with human learners, an describe the procedure and results of that experiment. The paper concludes with a discussion of the implications of these results for understanding the conditions under which the cultural ratchet can operate, and the kinds of cognitive capacities that teachers and learners need to possess in order to satisfy these conditions. Analyzing Cultural Transmission Our first step in exploring the conditions that result in cu- mulative cultural evolution is analyzing variants of iterated learning in which the learners are rational Bayesian agents. The assumption of rationality allows us to examine how ideal learners use the information transmitted between generations, and allows us to be explicit about the biases that influence learning. Specifically, we assume that all agents have a shared set of hypotheses H, and a prior distribution over these hy- potheses p(h) describing the degree to which each h ��� H is believed to be true in the absence of any evidence. Starting with these prior beliefs, agents observe the world and receive information from other agents, resulting in data,

hypothesis data data data hypothesis theory ... (c) ... theory theory hypothesis hypothesis (b) ... data data hypothesis hypothesis data (a) new data new data new data new data Figure 1: Three models of cultural evolution. (a) In the pure iterated learning model, each learner receives data from the previous learner and generates data provided to the next learner. (b) In the mixed data model, each learner also re- ceives data generated by the external world. (c) In the pos- terior passing model, each learner passes information about his or her current posterior distribution over hypotheses to the next learner (here taking the form of a theory). d. Using these data, each agent updates his or her beliefs by computing a posterior distribution p(h|d) using Bayes��� rule p(h|d) = p(d|h)p(h) ���h ���H p(d|h )p(h ) (1) where p(d|h) is the likelihood, expressing how probable the data are given a particular hypothesis, and the sum in the de- nominator ensures that the result is a normalized probability distribution. The learner can now choose a hypothesis based on its posterior probability. Intuitively, Bayes��� rule provides a way to combine prior beliefs with observations, with the probability assigned to each hypothesis being modified by the extent that it agrees or conflicts with the evidence. In the remainder of this section we describe three models of cultural evolution, varying in the nature of the data provided to learners. These three models are illustrated schematically in Figure 1. We analyze how the hypotheses selected by the learners change over time in each of these models, allowing us to determine whether the assumptions behind the model are sufficient to support cumulative cultural evolution. Pure Iterated Learning The first model might be called the pure iterated learning model, as it represents the standard scenario assumed in it- erated learning. In this scenario, each agent receives only data which are generated based on the hypothesis maintained by the previous agent. An intuitive example of this sort of process is the game ���telephone���, where each individual in a chain hears some message whispered by the previous person, and whispers what they think they heard to the next person. More formally, all agents share a prior p(h). There is some ���true��� hypothesis h* which is used to initiate the process. The first agent receives data d* sampled from the distribution as- sociated with this hypotheses, p(d*|h*). This agent���s beliefs are updated by applying Bayes rule (Equation 1) to reach a posterior p(h1|d*) ��� p(h1)p(d*|h1), samples a hypothesis h1 from this distribution, and produces data d1 by sampling from p(d1|h1).1 These data are passed to the second agent, and each subsequent agent similarly receives and transmits data. This process defines a Markov chain over hypotheses, with transition matrix p(hi|hi-1) = ���di-1 p(hi|di-1)p(di-1|hi-1). This chain has the prior p(h) as its stationary distribution, meaning that the probability a learner chooses a hypothe- sis h asymptotically converges to p(h) (Griffiths & Kalish, 2007). Laboratory experiments simulating this process of it- erated learning with human participants produce results that are consistent with this prediction (Kalish et al., 2007). An intuitive explanation for this convergence result can be provided by considering the ���telephone��� example. In this case, we expect that the original message tells us almost noth- ing about the message heard by a person far down the chain. This suggests that information is being lost over time. Loss of information allows the prior to influence the hypotheses selected by learners, who will use their a priori expectations to fill the gaps. Each generation, in reconciling the received data with their prior, thus passes less surprising data to the next generation. The prior itself is the only distribution over hypotheses that will be stable under this process. Mixed Data We can take a step towards producing cumulative cultural evolution by imagining a case where each person both re- ceives data from the previous person, and observes data from the external world. This is the mixed data case. Receiv- ing data from the world is necessary for cumulative cultural evolution, as agents are actually trying trying to learn about something external to their society, rather than a purely cul- tural creation. As an example, we might imagine that a hunter observes locations where antelope can be found grazing, but more experienced hunters also recommend specific locations where they would expect to find antelope. By combining these specific locations with prior beliefs about where ante- lope are found, the hunter reaches a new set of beliefs about where antelope are likely to be, and can tell other hunters about specific locations where he expects to find antelope. The formal description is a simple extension of the above model. All agents share some prior p(h). Each agent i still receives data di from the previous agent. Additionally, the ith agent receives some data di* from the world, gen- erated from p(di*|h*), where h* is the true hypothesis that 1All results given in this section still hold if learners sample data directly from the posterior predictive distribution, summing over all hypotheses, rather than sampling a hypothesis first.