Neural Language Models as What If? -Engines for HCI Research

Abstract

Collecting data is one of the bottlenecks of Human-Computer Interaction (HCI) and user experience (UX) research. In this poster paper, we explore and critically evaluate the potential of large-scale neural language models like GPT-3 in generating synthetic research data such as participant responses to interview questions. We observe that in the best case, GPT-3 can create plausible reflections of video game experiences and emotions, and adapt its responses to given demographic information. Compared to real participants, such synthetic data can be obtained faster and at a lower cost. On the other hand, the quality of generated data has high variance, and future work is needed to rigorously quantify the human-likeness, limitations, and biases of the models in the HCI domain.