The use of peer evaluations in a non-traditional first year system design class

Abstract

In the fall of 2010 the Chester F. Carlson Center for Imaging Science, an imaging systems engineering department at the Rochester Institute of Technology, completely abandoned its traditional lecture based pedagogy for incoming freshmen and in its place implemented a radically different project based class for first year students. Similar to many existing senior level capstone experiences, this new approach challenged first year students to work together as a single integrated multidisciplinary team for a full academic year to design, develop, build, and test a unique, fully functional imaging system from scratch. Now in its fourth year, all indications are that this pedagogy has been transformational, not only for the freshmen who have taken the class, but for the department as a whole. It has changed long held perceptions about the abilities of first year college students, and has led to a new understanding of the role of faculty in technical undergraduate degree programs. One of the central ways in which this pedagogy differs from a traditional approach is in its desired student outcomes. Whereas the outcomes of the old pedagogy were primarily knowledge oriented, in the new class the outcomes are focused on the degree to which students begin to adopt the behaviors and practices of professional engineers. Consequently conventional assessment tools which measure only knowledge, such as quizzes, tests, and final exams, are of limited value. Instead, instructors in the new class must rely on other techniques to assess student growth and development. One of these is the use of formal peer evaluations. Although these peer evaluations were treated as mandatory assignments due at the end of each academic term, their scope and format were determined by the students themselves. The evaluations were submitted to the instructor, who sanitized them to preserve the anonymity of the evaluators and then compiled and distributed them to each recipient. In this way every student received feedback on how at least a portion of their classmates perceived their performance three times over the course of the year long project. In this paper we examine the peer evaluations submitted by the first three cohorts to experience the new pedagogy in an effort to gain some insight into their use in a non-traditional classroom. Our analysis focuses on three primary aspects of the peer evaluation system. First, we look at how the scope and format of the evaluations evolved over time as a way of understanding which characteristics the students felt were most essential in their fellow team members. Next we examine the nature and quality of the feedback as a way to assess the perceived value of the peer evaluations. And lastly we draw from the records of the course instructor to see the degree to which the peer evaluations aligned with the perceptions of the instructors. Together these analyses can thus inform the use of peer evaluations as an assessment tool in engineering classes at other institutions. © American Society for Engineering Education, 2014.