Assessing science inquiry skills in an immersive, conversation-based scenario

Abstract

Innovative, interactive tasks that include conversations among humans and virtual (pedagogical) agents can be used to assess relevant cognitive skills (e.g., scientific inquiry skills). These new assessment systems aid the collection of additional information (e.g., timing data, information about conversation path sequences, and amount of help used) that provide the context for assessment and can inform assessment claims in these specific environments. In order to assess science skills, we have implemented and evaluated a game-like assessment with embedded conversations called the Volcano Scenario. This chapter describes the Volcano Scenario and highlights the techniques used to collect and analyze the data generated by the system. A hybrid approach to analyzing data from interactive, assessment environments that makes use of traditional psychometric analysis and several big data-related processes is described and illustrated through the analyses of data from 500 participants who have at least a year of college experience.