Student perspectives on cognitive engagement: Preliminary analysis from the course social and cognitive engagement surveys

Abstract

The following is a research paper centered around the discovery of the meaning of engagement to students and researchers. Increasing student in-class engagement remains a goal of the engineering education community, yet faculty continue to lack tools that allow them to measure their students' engagement. Development of tools surrounding engagement connects faculty to the best practices emergent from the research. Critical aspects of survey development include not only psychometric validity, but also shared contextual meaning among researchers, educators, and students. That is to say, instruments can have validity and reliability, but might not necessarily provide useful feedback to the faculty using them. The ways in which students' self-report is one way both faculty and researchers can make meaning of survey responses. As part of a larger research study, we used an innovative model to develop a survey tool to measure student's in-class cognitive engagement under Chi's Interactive/Constructive/Active/Passive (ICAP) framework. Students were included in the development process as a means of gaining understanding of their interpretation of survey items. We interviewed student survey participants, asking them to both explain what they believed the survey to be asking them and what actions shaped their responses. The purpose of this paper is to understand potential discrepancies between researcher intention and student interpretation of quantitative survey items. To that aim, we ask the following question: How do students interpret survey items related to in-class cognitive engagement? Preliminary findings suggest students' interpretation of items points to a discrepancy between researcher and student meaning of engagement. Though the survey was intended to target in-class engagement, students often conflated their in- and out-of-class engagement behaviors. Moreover, students did not distinguish between language we intended to reflect different levels of cognitive activity. As we continue to develop surveys surrounding engagement, this study gives useful insight into how we can interpret student responses and provide meaningful feedback to faculty. This is accomplished by understanding the ways in which researchers, faculty, and students talk about engagement differently, and how that might lead us towards shared meaning.