Using Programming Concept Inventory Assessments: Findings in a First-year Engineering Course

Abstract

This complete research paper examines the use of a programming concept inventory assessment in a first-year engineering course. At The Ohio State University, the first-year engineering program focuses on teaching introductory computer programming skills through MATLAB and C/C++. In recent years, this program has undergone curriculum changes which resulted in a desire to measure impacts on student learning. This led to a need for a validated assessment tool like a concept inventory which is used to assess students' conceptual understandings and misunderstandings. The use of concept inventories is common in STEM fields, specifically physics, and has been gaining popularity over the last 30 years. An existing language-independent programming concept inventory, SCS1, was replicated to create a MATLAB specific version (MCS1) in 2019-2020. Both the language-independent and MATLAB-specific assessment were given to students in autumn 2019 at this university. Because this assessment is given in a course with students with a wide range of programming experiences and different demographics, it was helpful to investigate the student results to see what differences exist in this population. This paper examines the demographic and prior programming experience information collected during the assessment, focusing on prior high school computer science experience, self-identified programming skill, gender, and honors vs. standard courses. Using independent samples t-tests, it was found that students who took a computer science course in high school were more confident in their skills but did not earn significantly higher scores on the assessments. While the average self-reported skill level for women was lower than that of men, there was no significant difference in assessment scores. Finally, it was also found that honors students performed significantly better on the assessments than standard students, however those differences are likely related to the extra programming instruction that those students received. The goal of this study is to provide insight into the programming skills of first-year engineering students with a variety of prior experience and perceived ability.