Analyzing Collaborative Interactions Across Domains and Settings: An Adaptable Rating Scheme

Abstract

In this chapter we report on the development of a rating scheme for the analysis of collaborative process data, and on its implementation in diverse CSCL settings. The rating scheme is composed of nine dimensions measuring ­different aspects of collaboration quality: sustaining mutual understanding, dialogue ­management, information pooling, reaching consensus, task division, time management, technical coordination, reciprocal interaction, and individual task orientation. It can be applied to recordings (video, audio, screen recordings, or log data) of student interaction and does not necessarily require transcripts or written records. While the rating scheme was originally developed in the context of a specific CSCL setting (video-based interdisciplinary problem-solving in the medical domain; Meier et al. (2007)), we demonstrate in our chapter that it can successfully be adapted to other CSCL settings. First, we introduce the initial rating scheme and its dimensions. Next we describe the process of adapting it to data from a very different CSCL setting (chat-based interaction in computer science classes). We briefly report on a study that used the ratings of collaboration quality as basis for adaptive feedback to students on how to improve their collaboration. Finally, we describe how we have integrated our rating scheme with ActivityLens (Avouris et al. 2007), a software tool which allows for a combined analysis of multiple sources of data (e.g., logfiles, audio and video recordings). Several tool modifications were made to permit analysis of collaborative process data from yet another CSCL study in which high-school students collaborated face-to-face on solving algebra problems with support from an intelligent tutoring system. We conclude our chapter with a discussion of practical implications for practitioners who may wish to adapt and apply our rating scheme.