A new assessment method to easily identify areas needing improvement in course-level learning outcomes

Abstract

One of the main purposes of the assessment process is to ensure continual improvement occurs in engineering programs. An analysis of the assessment process in the Department of Chemical Engineering at Brigham Young University revealed that the traditional method for assessing student learning from the student perspective never led to any such improvements. This portion of the assessment process consisted of the students rating, on a scale of 1-5, the course and themselves on each course learning outcome (termed competencies). The problem with this method, due to differences in the personal interpretations of the meaning of each number (e.g. 3 vs. 4) and the averaging of the data, was that the numerical data rarely indicated that problems existed. Moreover, no mechanism existed to uncover the reasons for the few instances where the rating was low. To improve the student portion of the assessment process, changes were made to 1) remove the numerical scale and replace it with a Yes/No question and 2) create a mechanism to determine the reasons for any low ratings. The former was done to help remove the numerical averaging of the data and reduce the variableness in individual interpretation of the meaning of the numbers in the 1-5 scale. The latter was done to help place the ratings in the proper context. The new assessment method consisted of two parts. Part 1 required the students to state whether or not they were proficient in each course competency (Yes/No). Part 2 required the students to briefly explain why competencies perceived as weak were so rated. The new assessment process was piloted in two courses during the Winter 2011 semester. The results easily identified problem areas in a manner not seen previously and did so without a significant increase in the class time needed to fill out the forms. Not only were weak competencies clearly distinguished from strong competencies, the reasons for the difficulties were also learned. Together, these two pieces of information proved to be a powerful combination in that they allowed the instructors to almost effortlessly identify specific changes that could be made to the course to improve student learning. The efficiency of the process is such that the faculty in the department recently approved a motion to adopt the new method across the entire curriculum. © 2012 American Society for Engineering Education.