How we know they're learning: Comparing approaches to longitudinal assessment of transferable learning outcomes

Abstract

This research paper describes interim results from a 4-year longitudinal study of how engineering students develop critical thinking, problem solving, and communication skills. The sample includes approximately 400 students in a mid-sized research intensive Canadian institution. The students were assessed using multiple approaches, including standardized tests, in-course activities, surveys, and course artefacts scored by a trained team using program-wide rubrics. Outcomes demonstrated in student course artefacts externally scored by VALUE rubric assessment increased over the two years. Scores on standardized tests generally trend upward with the Critical thinking Assessment Test (CAT) but are mixed on the Collegiate Learning Assessment (CLA+), most likely due to motivational and alignment issues. Student motivation is a significant issue in the project. The paper compares the assessment methods, and finds that using externally scored course artefacts is both less expensive and preferred by course instructors for course and program improvement over standardized tests.