Real-time generative AI experience of the Vanitas painting

Abstract

Generative Artificial Intelligence (GenAI) has become an important topic because of its great ability to generate human-like creative digital content, allowing the development of new experiences on different platforms such as metaverse, 3D scenarios, or virtual reality (VR). This allows us to create new immersive environments that enhance people′s interaction experiences. In this study, a suggestive immersive scenario for a neurohumanities experience is presented, where advanced technologies (i.e. face landmark detection, facial expression detection, image-to-image GenAI, and interactive video and sound) are combined to empower the user’s engagement through real-time monitoring and interaction. To obtain feedback on the human emotional response, electroencephalography (EEG) and biometric signals (electrodermal activity, EDA) were analyzed in the time and frequency domains. The general results suggest the impact of immersive scenario interaction in the frequency domain using power spectral density (PSD) compared to the baseline state that was measured prior to the test.