JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 46, NO. 6, PP. 716���730 (2009) Measuring Progressions: Assessment Structures Underlying a Learning Progression Mark Wilson Graduate School of Education, UC Berkeley, Berkeley, California 94720 Received 24 June 2008 Accepted 1 May 2009 Abstract: This article describes some of the underlying conceptualizations that have gone into the work of the BEAR Center in the development of learning progressions. The core of all of these developments has been the construct map, which is the first building block in the BEAR Assessment System (BAS). After introducing the concept of a learning progression, the article summarizes the elements of the BAS, emphasizing the central concept of a construct map. The article then describes a series of several different ways to see the relationship between the idea of a construct map and the idea of a progression (which I call the ������assessment structure������), and also gives illustrative examples from recent BEAR projects. The article then discusses some strengths and limitations of these conceptualizations, focusing on both educational and measurement issues. The article concludes with some general reflections. �� 2009 Wiley Periodicals, Inc. J Res Sci Teach 46: 716���730, 2009 Keywords: general science learning progressions assessment structure middle school science The idea of a learning progression is one that is undergoing swift development at the current time. However, it is really just the latest manifestation of a much older idea, that of regularity in the development of students as they learn a certain body of knowledge or professional practice. Devising means of measuring a student���s location within or along a learning progression is a crucial step in advancing the scientific study of learning progressions, and for finding educationally useful applications of the idea. In this article, one particular approach to measurement, called the BEAR Assessment System (BAS Wilson, 2005 Wilson & Sloane, 2000), is used as a lens through which to portray a perspective on themanypossibleways that learning progressions could be conceived of and measured. In this article, the manner in which the measurement approach supports the learning progression is referred to as the assessment structure for the learning progression. Of course, there are other measurement approaches that one could take, but these are outside the scope of this current effort. The article beginswith a brief note about the concept of learning progressions and adds some notes about assessment perspectives on learning progressions. It then summarizes the elements of the BAS, emphasizing the central concept of a construct map, which is the focus of the rest of the article. It then describes a series of several different ways to see the assessment structures���the relationship between the idea of a construct map and the idea of a progression���and gives examples from current and recent BEAR work. It then discusses the strengths and limitations of these conceptualizations, focusing on both educational and psychometric issues. The article concludes with some general reflections. Learning Progressions: Links to Assessment In general, this articlewill follow the definition of learning progressions as given in the lead article in this issue (Duncan & Hmelo-Silver, 2009). The purpose of the current article is to attempt to lay out some possible patterns of relationships between learning progressions and a concept that has been developed within a measurement and assessment framework, the concept of the construct map. This will be defined in some Correspondence to: M. Wilson E-mail: markw@berkeley.edu DOI 10.1002/tea.20318 Published online 7 July 2009 in Wiley InterScience (www.interscience.wiley.com). �� 2009 Wiley Periodicals, Inc.

detail in the next section of the article, but suffice it to say at this point that a construct map is intended to be a somewhat less complex concept than a learning progression, and is designed to help conceptualize how assessments can be constructed to relate to theories of cognition. Although the idea of a learning progression has links to many older and venerable ideas in education, the history of the specific term ������learning progression������ in the context of science education is a relatively brief one (CCII, 2009), starting with the publication of an NRC report (2006). That report was focused on assessment in K-12 education, and hence the connections to assessment have been there right from the start. Nevertheless, given the brief time-span since then, there is not a great deal of extant literature regarding the relationship between the two, although this may well change in the near future. A second NRC report (2007) also featured the concept, and enlarged upon classroom applications. Several assessment initiatives and perspectives are discussed in these reports, including references to the seminal 2001 NRC report Knowing What Students Know. Among the assessment programs highlighted there, probably the most prominent is the work on progress variables by the Australian researchers Masters and Forster (1996), and the closely related work on the somewhat more elaborated BAS (Wilson, 2005). In this article, I will draw on the latter as the core set of assessment perspectives and practices to relate to learning progressions. In order to illustrate certain aspects of the relationship between learning progressions and assessment, I will use a visual metaphor that superimposes images of construct maps on an image of a learning progression. This image of the learning progression is shown in Figure 1, where the successive layers of the ������thought clouds������ are intended to represent the successive layers of sophistication of the student���s thinking, and the increase in the cloud���s size is intended to indicate that the thoughts become more sophisticated later in the sequence (e.g., they have wider applicability later in the sequence). The person in the picture is a someone (a science educator, a science education researcher, an assessment developer?) who is thinking about student thinking. In other circumstances (e.g., Wilson, 2005), I have called this person the ������measurer,������ though not here, as the ideas being examined in the article are mainly at an early point in the development of assessments, focusing on the first of the building blocks. It is important to recall that this learning progression is in the researcher���s thoughts, and that it represents a hypothesis about the students��� thoughts that will be examined empirically, eventually. Figure 1. An image of a learning progression. MEASURING PROGRESSIONS 717 Journal of Research in Science Teaching

The BEAR Assessment System The BAS is based on the idea that good assessment addresses the need for sound measurement through four principles: (1) a developmental perspective, (2) a match between instruction and assessment, (3) the generating of quality evidence, and (4) management by instructors to allow appropriate feedback, feed forward and follow-up. These four principles, plus four building blocks that embody them, are shown in Figure 2. Below we take up each of these principles and building blocks in turn, emphasizing the first. See Wilson (2005) for a detailed account of an instrument development process that works through these steps. Principle 1: A Developmental Perspective A ������developmental perspective������regarding student learning means assessing the development of student understanding of particular concepts and skills over time, as opposed to, for instance, making a single measurement at some final or supposedly significant time point (for earlier perspectives on this see Hewson, 1992 and Posner, Strike, Hewson, & Gerzog, 1982). Establishing appropriate criteria for taking a developmental perspective has beena challenge toeducators for many years. What toassess and how to assess it, whether to focus on generalized learning goals or domain-specific knowledge, and the implications of a variety of teaching and learning theories all impact what approaches might best inform developmental assessment. One issue is that as learning situations vary, and their goals and philosophical underpinnings take different forms, a ������one-size-fits-all������ development assessment approach rarely satisfies educational needs. Much of the strength of the BAS comes in providing tools to model many different kinds of learning theories and learning domains. What is to be measured and how it is to be valued in each BEAR assessment application is drawn from the expertise and learning theories of the teachers, the curriculum developers, and the assessment developers involved in the process of creating the assessments. Building Block 1: Construct Maps. Construct maps (Wilson, 2005) embody this first of the four principles: that of a developmental perspective on assessment of student achievement and growth. A construct map is a well thought out and researched ordering of qualitatively different levels of performance focusing on one characteristic. Thus, a construct map defines what is to be measured or assessed in terms general enough to be interpretablewithin a curriculum and potentially across curricula, but specific enough to guide the development of the other components. When instructional practices are linked to the construct map, then the construct map also indicates the aims of the teaching. Construct maps are one model of how assessments can be integrated with instruction and accountability. They provide a way for large-scale assessments to be linked in a principled way to what students are learning in classrooms, while at least having the potential to remain independent of the content of a specific curriculum. Figure 2. The principles and building blocks of the BEAR Assessment System. 718 WILSON Journal of Research in Science Teaching