Story quality as a matter of perception: Using word embeddings to estimate cognitive interest

Abstract

Storytelling is a capable tool for interactive agents and better stories can enable better interactions. Many existing automated evaluation techniques are either focused on textual features that are not necessarily reflective of perceived interestingness (e.g. coherence), or are domain-specific, relying on a priori semantics models (e.g. in a game). However, the effectiveness of storytelling depends both on its versatility to adapt to new domains and the perceived interestingness of its generated stories. In this paper, drawing from cognitive science literature, we propose and evaluate a method for estimating cognitive interest in stories based on the level of predictive inference they cause during perception.