Collecting, Analyzing, and Interpreting Response Time, Eye-Tracking, and Log Data

Abstract

This chapter argues for a distinction between protocol analyses and verbal analyses, with the former being used to confirm models for problem-solving tasks and the latter used in a more exploratory mode to study comprehension. Verbal response data may be used to assess comprehension of the problem to be solved as well as to indicate how the examinee characterizes and manipulates the problem space. The chapter discusses the use of non-verbal response process data at different stages of test development, from pre-concept through operational use. Timing data is also a useful component of data forensics efforts to construct indicators of various forms of cheating. Log data include both keystroke information and also data on a variety of other test-taker actions, particularly when touch screens, mouse movement and clicks, or a variety of other input devices are involved. Retaining gigabytes of data for each student and transmitting it to a central location could be quite daunting.