Applications and Modeling of Keystroke Logs in Writing Assessments

Abstract

In this paper, we describe two empirical studies that demonstrate the application and modeling of keystroke logs in writing assessments. We illustrate two different approaches of modeling differences in writing processes: analysis of mean differences in handcrafted theory-driven features and use of large language models to identify stable personal characteristics. In the first study, we examined the effects of test environment on writing characteristics: at-home versus in-center, using features extracted from keystroke logs. In a second study, we explored ways to measure stable personal characteristics and traits. As opposed to feature engineering that can be difficult to scale, raw keystroke logs were used as input in the second study, and large language models were developed to infer latent relations in the data. Implications, limitations, and future research directions are also discussed.