Measuring User Experience on Massive Open Online Course with User-Centered Metrics: A Case Study

Abstract

Evaluation of user experience (UX) in learning applications is now very important. Agood UX on MOOC will have an impact on increased completion rates and student satisfaction,student engagement, enhanced learning, and minimizing course implementation rework. In thisstudy, the authors use MOOC called E-learning: Open of Knowledge Sharing (eLOK) as theobject of research. User-centered metrics or known as HEART framework consisting of fivemetrics, including Happiness, Engagement, Adoption, Retention and Task Success. Theframework that the authors used to measure the UX of eLOK is HEART metrics. The instrumentthat used in this study is a questionnaire consists of 33 statements which delivered to 94 studentsas the sample of the research, with the the number of population 1500. The questionnaire hasbeen validated using SPSS 23 as the tools. The result of the validation is all the items of thequestionnaire are valid and reliable with the percentage of reliability is 96%. The result of thisstudy showed that the metrics obtained by Happiness was 76.5%, Engagement was 72.1%,Adoption was 73.6%, Retention was 72.6%, and Task Success was 78.6%. To achieve the GoalSignal-Metrics set on the HEART framework, it is necessary to re-design the eLOK display bypaying attention to UX and UI aspects, changing the greeting with user-guidance, improving pushnotification features and navigation.