Reference frames for learning analytics dashboards: The progress and social reference frame and occupational self-efficacy

Abstract

Background: The potential of learning analytics dashboards in virtual reality simulation-based training environments to influence occupational self-efficacy via self-reflection phase processes in the Chemical industry is still not fully understood. Learning analytics dashboards provide feedback on learner performance and offer points of comparison (i.e., comparison with one's own past performance or comparison with peer performance) to help learners make sense of their feedback. Objectives: We present a theoretical framework for describing learning analytics reference frames and investigate the impact of feedback delivered through dashboards with different reference frames on occupational self-efficacy, while controlling for workplace self-reflection. Methods: This experimental study engaged 42 chemical operator employees, aged between 18 and 55 years, each with at least one year of experience. We utilised a two-group design to ask two research question each with three competing hypotheses related to changes in occupational self-efficacy, employing Bayesian informative hypothesis evaluation. Results and Conclusions: Results for the primary research question suggest that dashboards with progress reference frames do not elicit greater change to self-efficacy than those with social reference frames, however, they may elicit equal change. Furthermore, dashboards with social reference frames may elicit greater change to self-efficacy than those with progress reference frames. Exploratory results found that dashboards with progress reference frames may elicit greater positive directional change than those with social reference frames and that they may elicit equal directional change. These findings contribute to the understanding of self-efficacy beliefs within the Chemical industry, with potential impacts on skill development. The research may inform the design of targeted interventions and training programs to influence self-efficacy. From a practical perspective, this research suggests that careful consideration is needed when choosing reference frames in learning analytics dashboards due to their potential consequences on the formation of learner self-efficacy.