Requirements monitoring and diagnosis for improving adaptive e-learning systems design

Abstract

Aim/Purpose In this paper, we highlight the need to monitor and diagnose adaptive e-learning systems requirements at runtime to develop a better understanding of their behavior during learning activities and improve their design. Our focus is to reveal which learning requirements the adaptive system is satisfying while still evolving and to provide specific recommendations regarding what actions should be taken and which relevant features are needed to help meet the speci-fied learning requirements. Background Adaptive e-learning systems research has long focused on user modeling and social learning to personalize each learner experience, while fewer instruments are reported to assess the quality of the solutions provided by such adaptive systems and to investigate their design problems. The design problems may emerge due to ever-evolving requirements being statically specified at design stages and to the changing environments that can be difficult to control and observe. The combination of some or all of these factors can lead to a defini-tion of inconsistent or insufficient adaptation rules, which in turn may prevent these systems from providing appropriate resources to learners even if the needed ones have been accounted for within the knowledge space. Methodology An empirical study has been performed to check and validate the behavior of a real-world adaptive e-learning system under four stated requirements. The study used a novel monitoring and diagnosing tool that reads the collected data from the system and checks its behavior against constraints that are derived automati-cally from the requirements specification. Contribution The results provide statistical insights and highlight some issues related to re-quirements compliance at runtime, which helped us detect unforeseen instruc-tional design issues. Recommendations for Practitioners The study suggests that diagnosing requirements compliance at runtime can be an essential means to increase the confidence about their adaptive e-learning systems capabilities at runtime. Recommendations for Researchers The study suggests that further research for developing specific indicators relat-ed to requirements compliance is needed in the field of adaptive e-learning systems. Future Research Future work includes the study of possible improvement of our diagnostic tool using probabilistic reasoning.