Training Structural Awareness with StatHand: A 1 Year Follow-Up

Abstract

Background: Allen, Fielding, East, et al. demonstrated experimentally that structural awareness, or the ability to disregard a research problem’s topic and instead focus on its structural features, can be trained using StatHand (https://stathand.net). Most training benefits persisted for 1 week. Objective: The objective was to assess the longer-term effects of training. Method: One year after training (or control activities), 54 participants were re-administered 5 measures of structural awareness and 1 statistic selection measure. Results: Trained participants continued to reliably out-perform control participants on 4 measures of structural awareness, though no longer on the 5th. Over the year, decrements in trained participants’ performance on the 5 structural awareness measures were mostly small. However, 1 year after training, the trained participants’ statistic selection advantage had largely disappeared. Conclusion: Brief structural awareness training can have long-term benefits, though selecting an appropriate statistical test for common research scenarios without assistance remains a difficult task. Teaching Implications: Structural awareness can be trained. However, even structurally aware students cannot reliably select appropriate statistics without assistance. Training plus easy access to a decision-making aid should maximize statistic selection accuracy. Our evidence-based training methods and materials, including StatHand, can be freely used and adapted for these purposes.