Measuring student content knowledge, iSTEM, self efficacy, and engagement through a long-term engineering design intervention

Abstract

The current study reports on the outcomes of a classroom-based long-term engineering design intervention intended to increase high school students' perceptions of the integrated nature of STEM disciplines (iSTEM) and to assess the effect of the intervention on student participation in an extracurricular STEM activity (i.e., a research poster symposium). Cross-disciplinary teams of students (n=373) from high school mathematics, science, and engineering classrooms completed engineering design challenges. The results indicated that, consistent with our predictions, the intervention exhibited a positive impact on students that began the study with the lowest iSTEM scores. Furthermore, the classroom environment mattered. While no individual scores (i.e., posttest iSTEM scores) were predictive of participation in the poster symposium, the collective scores were (i.e., mean classroom iSTEM scores). Four measures were used in this study; Content knowledge quiz. Student content knowledge was assessed with a teacher made nine-item multiple-choice quiz; self-efficacy, task specific self-efficacy was assessed through a nine item measure; iSTEM perceptions. Participants responded to a nine-item iSTEM scale developed and validated by the authors in a previous study, to measure student perceptions of the interconnections between mathematics, science, and engineering; and STEM clubs. Participants responded "Yes" (1) or "No" (0) to the question regarding their involvement in extracurricular STEM club. Hierarchical linear modeling (HLM) was used in this analysis because it distinguishes variability in scores at the student-level (i.e., level-1) from variability in scores at the classroom level (i.e., level-2), which results in correctly estimating standard error. Therefore, HLM was used to conduct multilevel-paired sample t-tests. Further, all analyses were conducted with Restricted Maximum Likelihood estimation. The results indicated that, consistent with our predictions, the intervention exhibited a positive impact on students that began the study with the lowest iSTEM scores. Furthermore, the classroom environment mattered. While no individual scores (i.e., posttest iSTEM scores) were predictive of participation in the poster symposium, the collective scores were (i.e., mean classroom iSTEM scores).