Correlation of online assessment parameters with summative exam performance in undergraduate medical education of pharmacology: a prospective cohort study

Abstract

Background: Learning analytics aims to improve learning outcomes through the systematic measurement and analysis of learning-related data. However, which parameters have the highest predictive power for academic performance remains to be elucidated. The aim of this study was to investigate the correlation of different online assessment parameters with summative exam performance in undergraduate medical education of pharmacology. Methods: A prospective study was conducted with a cohort of undergraduate medical students enrolled in a pharmacology course at Technical University of Munich, Germany. After a four-week teaching and learning period, students were given access to an online assessment platform consisting of 440 multiple choice (MC) questions. After 12 days, a final written summative exam was performed. Bivariate correlation and multiple regression analyses were performed for different online assessment parameters as predictors and summative exam performance as dependent variable. Self-perceived pharmacology competence was measured by questionnaires pre- and postintervention. Results: A total of 224 out of 393 (57%) students participated in the study and were included in the analysis. There was no significant correlation for the parameters "number of logins" (r = 0.01, p = 0.893), "number of MC-questions answered" (r = 0.02, p = 0.813) and "time spent on the assessment platform" (r = - 0.05, p = 0.459) with exam performance. The variable "time per question" was statistically significant (p = 0.006), but correlated negatively (r = - 0.18) with academic performance of study participants. Only "total score" (r = 0.71, p < 0.001) and the "score of first attempt" (r = 0.72, p < 0.001) were significantly correlated with final grades. In a multiple regression analysis, "score first attempt" accounted for 52% of the variation of "score final exam", and "time per question" and "total score" for additional 5 and 1.4%, respectively. No gender-specific differences were observed. Finally, online assessments resulted in improved self-perceived pharmacology competence of students. Conclusion: In this prospective cohort study, we systematically assessed the correlation of different online assessments parameters with exam performance and their gender-neutrality. Our findings may help to improve predictive models of academic performance in undergraduate medical education of pharmacology.