Stealth Literacy Assessment: Leveraging Games and NLP in iSTART

Abstract

Literacy skills are critical to academic success and in life, and literacy assessments have been used extensively to help improve students’ skills. In this chapter, we propose an innovative implementation of natural language processing (NLP) to conduct stealth assessment of literacy skills. We describe literacy assessments, including applications of NLP techniques to assess literacy. We also provide an overview of stealth assessment and its application in digital environments to assess students’ knowledge and skills. Stealth literacy assessment is introduced in the context of Strategy Training for Active Reading and Thinking (iSTART), a game-based intelligent tutoring system designed to help students improve their reading comprehension. Two analyses leveraging secondary data demonstrate that the linguistic properties of student-constructed responses successfully predict reading skills, with more accurate predictions using linguistic features of responses constructed during iSTART training as compared to students’ responses during independent self-explanation assessments. Overall, the chapter offers a significant step forward in demonstrating the feasibility of stealth literacy assessments leveraging NLP.