Understanding context: Propagation and effectiveness of the concept warehouse in mechanical engineering at five diverse institutions and beyond - Results from year 1

Abstract

Several consensus reports cite a critical need to dramatically increase the number and diversity of STEM graduates over the next decade. They conclude that a change to evidence-based instructional practices, such as concept-based active learning, is needed. Concept-based active learning involves the use of activity-based pedagogies whose primary objectives are to make students value deep conceptual understanding (instead of only factual knowledge) and then to facilitate their development of that understanding. Concept-based active learning has been shown to increase academic engagement and student achievement, to significantly improve student retention in academic programs, and to reduce the performance gap of underrepresented students. Fostering students' mastery of fundamental concepts is central to real world problem solving, including several elements of engineering practice. Unfortunately, simply proving that these instructional practices are more effective than traditional methods is not enough to ensure widespread pedagogical change. In fact, the biggest challenge to improving STEM education is not the need to develop more effective instructional practices, but to find ways to get faculty to adopt the evidence-based pedagogies that already exist. In this project we seek to propagate the Concept Warehouse (CW), a technological innovation designed to foster concept-based active learning, into Mechanical Engineering (ME) and to study student learning with this tool in five diverse institutional settings. The CW is a web-based instructional tool that we developed for Chemical Engineering (ChE) faculty. It houses nearly 3,000 ConcepTests, which are short questions that can rapidly be deployed to engage students in concept-oriented thinking and/or to assess students' conceptual knowledge, along with more extensive concept-based active learning tools. The CW has grown rapidly over the last four years (around 1,200 faculty accounts and 30,000 student users). We propose to expand use of the CW into mechanics and ME and thereby impact 50,000 students during this project. Although the current CW content is discipline-specific, the functions are generic and readily transferable to other engineering disciplines as content is developed. To date, our Statics and our Dynamics Teams have developed 107 and 105 new ConcepTests, respectively. Question development and categorization has included a framework developed by Beatty et al. (2006), and includes utilizing Content, Process, and Epistemological Goals for each question. Beta testing with students is currently being conducted, and includes questions on both clarity and on educational effectiveness. Our goal is to create 150 ConcepTests in each subject by the time of the conference. Twelve instructors from the partner schools have been recruited to help us study how context affects the adoption of the Concept Warehouse. These Phase I participants were asked to use the CW to deploy a Concept Inventory, and were then interviewed to examine the instructors' perceptions of their institutional and learning context and their histories and beliefs. Phase II participants will be asked to deploy ConceptTests within their classrooms, and site visits will be conducted for additional interviews, classroom visits, and student focus groups. These will be used in conjunction with institutional context at five very different schools (a large research public university, a small private university, a 2-year college serving a large number of underrepresented students, a large non-PhD granting public university, and a bilingual research university) to determine the conditions that are most supportive of adopting educational innovations.