Immersive Conversations with Digital Einstein: Linking a Physical System and AI

Abstract

We present Digital Einstein, an immersive platform that enables interactive conversations with a digital representation of Albert Einstein. The physical setup includes a screen, armchair, camera, speakers, microphone, and decorative elements. Utilizing advanced technologies such as natural language processing, machine learning, and dynamic rendering, the platform simulates realistic dialogs with Einstein on various topics about his life and research. The system features two versions: Version 1 uses a dialog tree, motion-captured animations, and pre-recorded speech for fluid, controllable interactions. In Version 2, users can choose between GPT-4o-based and Llama 3-based chatbots for more open and engaging conversations. Data-driven models synthesize Einstein's voice and generate corresponding animations on the fly, while context-specific images are automatically generated to visually enrich the conversation. Digital Einstein offers a glimpse into the future of immersive, personalized dialog experiences with intelligent characters.