A design experiment on technology-based learning progress feedback in a graduate-level online course

Abstract

Our literature review has revealed that the definition of learning progress as a function of formative feedback remains unclear, which continues to the lack of well-reasoned learning progress feedback (LPF) strategies. Also, few scholars have empirically examined student perceptions of informed learning progress in solving a complex problem. The current study aims (a) to generate insights that elaborate LPF in a technology-enhanced learning environment and (b) to gain better knowledge about LPF design for technology-enhanced formative assessment and feedback. In this design experiment, we proposed LPF design strategies in five categories. Then, we experimented with two LPF prototypes to see how students reacted to the LPF features and how students perceived the value of LPF feedback. In the context of a graduate-level online course with 35 student participants, we performed a mixed-data analysis. Results indicate that students benefited from the two LPF designs to navigate their learning progress. Students' LPF preference appeared to relate to their feedback literacy regarding revision behavior, cognitive progress, and emotional reaction. The current findings lay the groundwork for elaborate LPF models that are pedagogically effective and adaptive to diverse student needs. The proposed LPF design typology and further design suggestions can help advance LPF design experiments and improve technology-based formative feedback that enables students to pursue mastery.