PathOS+: A new realm in expert evaluation

Abstract

Expert evaluation is commonly employed in usability research as it is fast and cost-effective. However, as it heavily relies on evaluators' expertise, it is associated with problems of subjective interpretation. This is particularly noticeable in the games user research (GUR) field as games often aim to reach broad audiences and complex experiences in comparison to other applications. One way to improve the problems associated with subjective interpretation is to augment the evaluation with data collected from players. However collecting data from players can be time-consuming, expensive, and challenging to construct representative player samples. Building on our previous research on automated agent-based playtesting, in this work-in-progress paper we detail our new project in supplementing expert evaluation with simulated player data generated by a configurable population of artificial intelligence (AI) players.