What instruments do researchers use to evaluate LXD? A systematic review study

Abstract

In contrast to traditional views of instructional design that are often focused on content development, researchers are increasingly exploring learning experience design (LXD) perspectives as a way to espouse a broader and more holistic view of learning. In addition to cognitive and affective perspectives, LXD includes perspectives on human–computer interaction that consist of usability and other interactions (ie—goal-directed user behavior). However, there is very little consensus about the quantitative instruments and surveys used by individuals to assess how learners interact with technology. This systematic review explored 627 usability studies in learning technology over the last decade in terms of the instruments (RQ1), domains (RQ2), and number of users (RQ3). Findings suggest that many usability studies rely on self-created instruments, which leads to questions about reliability and validity. Moreover, additional research suggests usability studies are largely focused within the medical and STEM domains, with very little focus on educators' perspectives (pre-service, in-service teachers). Implications for theory and practice are discussed.