Successful use of rubrics to assess student performance in capstone projects

Abstract

Capstone Experience, MTC 420, is a required course for all Mechanical Engineering Technology (MET) students during their senior year. The capstone projects are intended to be culminating experiences, drawing upon a wide range of knowledge from courses in the curriculum. Students are responsible for written project specifications, planning and milestone identification, implementation of the work, an oral presentation, and a final written report. Because these projects are so comprehensive, they provide an opportunity for faculty to assess a wide range of student learning that is directly related to program outcomes. For this reason, MET faculty developed a rubric for assessing capstone projects, as shown on the following page. The instructor rates each project in terms of ten attributes on a scale of one to four, providing a direct measure for assessment of program outcomes. The scale of one to four (1-Not Acceptable, 2-Below Expectations, 3-Meets Expectations, 4-Exemplary) was intentionally chosen so that the instructor had to make a clear decision as to whether or not expectations were met. The rubrics were administered at the end of each semester, fall 2007 through spring 2009, and results were tabulated and analyzed to identify areas which were satisfactory and areas needing improvement. The measure of successful performance in each attribute was the percentage of students achieving a level of three or four. A benchmark of 70% was used to gage the level of success. If 70% of students scored a three or four, then the program was considered successful for that attribute. An interesting side-effect occurred when students were given copies of the rubrics at the beginning of the semester in fall 2008. This raised their awareness of expectations, especially in the non-technical areas such as teamwork and communication skills. A simpler rubric was also used to evaluate students' mid-semester oral presentations. After the presentation, the instructor completed the rubric for each student using a binary scale (0-unsuccessful, 1-successful). The measure of satisfactory performance in each category was the percentage of students scoring "1." A benchmark of 70% was used to gage the overall level of success. If 70% of the students were successful, then the presentations were considered successful. Results from this rubric gave the instructor and students feedback, and the results also provided a basis for monitoring improvements that occurred during the second half of the semester. The levels of success in the capstone rubric were documented and correlated with levels of achievement for six program outcomes for four consecutive semesters. In fall 2009, these results were accepted by TAC/ABET as part of the continuous improvement program in MET. Note that the rubrics were not used directly to assign grades; however, results gave the instructor useful insight for evaluating the projects and assigning grades. Although the rubrics took some initial effort to develop, it is now fairly easy to complete and analyze the results each semester as a routine part of the ongoing assessment processes. © American Society for Engineering Education, 2010.