Real-Time Affective Measurements in Medical Education, Using Virtual and Mixed Reality

Abstract

Emotion greatly affects learning. Affective states, such as motivation, interest and attention, have been identified to cause changes in brain and body activity. Heart Rate (HR), Electro-Dermal Activity (EDA), and Electroencephalography (EEG) reflect physiological expressions of the human body that change according to emotional changes. In reverse, changes of bio-signal recordings can be linked to emotional changes. Virtual/Mixed Reality (V/MR) applications can be used in medical education to enhance learning. This work is a proof of application study of wearable, bio-sensor based affect detection in a learning processes, that includes the Microsoft HoloLens V/MR platform. Wearable sensors for HR, EDA and EEG signals recordings were used during two educational scenarios run by a medical doctor. The first was a conventional scenario-based Virtual Patient case for the participant’s bio-signal baselines canonization. The second was a V/MR exploratory educational neuroanatomy resource. After pre-processing and averaging, the HR and EDA recordings displayed a considerable increase during the V/MR case against the baseline. The alpha rhythm, of the EEG, had a borderline degrease and the theta over beta ration a borderline increase. These results indicate an increased attention/concentration state. They also demonstrate that the usage of bio-sensors assist in the detection the emotional state and could provide real-time, affective learning analytics using V/MR in medical education.