Within, Between, Forced Choice, or Likert Scale? How Methodological Decisions Influence Recognition Rates in HRI Recognition Studies

Abstract

Recognition studies are a mainstay in HRI. Such studies are often used to test whether a robot’s behavior is interpreted as intended by the designer. When designing recognition studies, researchers have to make important methodological decisions about the empirical study design (e.g., within-/between-subject design) and response format (e.g., forced choice, open text). Using the example of emotional expression recognition studies in HRI, we discuss (i) how theoretical conceptualizations determine methodological choices, (ii) the implications of the designs and response formats. We demonstrate in two experiments (n = 2654 and n = 419) that conclusions drawn from recognition studies are heavily dependent on study design and response format. We conclude with a set of recommendations for researchers employing recognition studies in their research.