Utilizing software-generated concept maps based on customized concept inventories to illustrate student learning and knowledge gaps

Abstract

Concept inventories have been developed for a variety of disciplines over the last 20 years in order to evaluate student understanding of subjects within the discipline at the conceptual level. Concept inventories have served as a basis for evaluation of student's fundamental understanding of topics and students' responses to concept inventories showcase areas where there is need for enrichment. Concept maps are directed graphs that depict the relationships within a discipline and have been used since the 1970s to showcase how knowledge is constructively assembled within a learner's cognitive structure. Concept maps graphically depict the concepts and relationships within a discipline. Typically, concept maps are constructed by students in order to allow evaluators to understanding learners' comprehension of the relationships that exist within a discipline. In this paper software is described that has been developed to couple concept inventories with concept maps in order to elicit graphical representations of students' understanding of a discipline based upon their responses to a custom-designed concept inventory. This software allows for entry of the questions and answers within a concept inventory and allows each answer to be correlated with a concept or relationship in a concept map. Under this system an expert develops a comprehensive concept map that depicts all of the underlying concepts and relationships within a discipline and then the questions in the concept inventory are tied to the appropriate portions of the concept map. The results of the deployment of the inventory are then depicted in a graphical student map showcasing each individual learner's mastery of the underlying concepts and relationships as a concept map that is a subset of the comprehensive map. The software also generates aggregate maps based upon various demographics so that performance can be examined on both a student-by-student basis and also at a holistic higher level. This software has been used in conjunction with a course unit on mechatronics designed to introduce first-semester engineering students to the fundamentals of mechanics and electronics. A comprehensive map has been developed and a corresponding concept inventory has been designed to test student's knowledge gains through participation in the course unit. The concept inventory has been given as a pre and post test and has also been given in a subsequent semester to evaluate student retention of conceptual knowledge over time. Over 1400 data sets have been collected and analyzed. Within this paper the components of the analysis software are discussed along with some results from data analysis. © American Society for Engineering Education, 2010.