How to use simulation in the design and evaluation of learning environments with self-directed longer-term learners

Abstract

Designing, developing, and evaluating interactive and adaptive learning environments requires significant investment of financial and human resources. This is especially the case when evaluating the impact of learning environments aimed at supporting self-directed longer-term learners, environments of increasing interest to AIED as the field moves into lifelong learning where mentorship is key. In this paper we propose the use of simulation to help in both the design and evaluation of such environments. As a case study, we have built a simulated university doctoral program (SimDoc) with simulated doctoral students and supervisors (mentors). To make sure SimDoc replicates observed data from a real-world environment as closely as possible, we informed and calibrated the simulation model with data from an actual doctoral program, as well as drawing on various empirical studies of graduate students and supervisors. Next, we used the calibrated simulation model to explore the effect of varying research group sizes of learners and supervisors’ mentoring workload on students’ completion rates and time-to-completion. Our main goal is to provide insight into how to build simulations of environments that support self-directed longer-term learners through a case study of one such simulation, thus further demonstrating the importance of simulation in AIED.