The Creation of Immersive Experiences in Transcultural Entertainment: An Action Design Process Focused on Neural Rendering

Abstract

This study examines how artificial intelligence (AI), specifically neural facial reenactment (NFR), can address the limitations of dubbing and subtitling in adapting foreign films. Using action design research and guided by presence theory, we codeveloped and evaluated an NFR process during the English adaptation of the Polish feature film The Champion. Unlike conventional dubbing, which often disrupts immersion through poor lip-sync or script changes, NFR preserves the original actors’ performances, aligning them with new dialogue. Independent broadcast-quality assessments confirmed technical validity, and subsequent commercial distribution on a major streaming platform demonstrated scalability and audience acceptance. From this process, we derived six design principles: avoid forced script changes, respect creative intent, minimize intrusive technology, reduce training data requirements, enable flexible audience access, and codesign with existing creative structures. These principles offer a replicable template for the responsible researching and developing of AI in information systems. For practice, the findings show that NFR can improve cultural accessibility and create new creative and technical roles rather than displacing talent. For policy, the study highlights the importance of codesign, transparency, and preserving artistic integrity when integrating AI into global cultural products.We conceptualize the emergent use of artificial intelligence to craft immersive cross-cultural experiences in media and entertainment (M&E). The class of problems that we are looking to address is the creation and maintenance of an immersive experience with presence when an M&E organization is transposing a visual entertainment product across languages and cultural contexts. We conducted an action design research study using neural facial reenactment (NFR) technology to visually adapt foreign film actors’ facial performances. This study is significant for its potential to revolutionize foreign film adaptation, surpassing traditional dubbing and subtitling. We engaged critically with data collected during the building, intervention, and evaluation adaptation of the Polish film The Champion. With presence theory as our guiding theoretical framework, we examine how NFR technology can enhance emotional engagement and narrative authenticity by facilitating spatial, social, and self-presence. The primary artifact developed through this research is the NFR process we adapted, and informed by presence theory, a set of six design principles related to the artifact were developed. The six design principles with broader information systems applications in transcultural communications, education, and virtual assistants, include (1) avoiding forced script changes, (2) respecting creative intent in translation, (3) minimizing intrusive technology, (4) reducing training data needs, (5) enabling informed flexible audience access, and (6) codesigning within existing structures. From this, we advance three distinct and validated claims. First, unlike conventional techniques, NFR demonstrably preserves alignment with the original creative intent. Second, the process demonstrably enhances production quality, remaining feasible within industry constraints. Third, the application of these principles in multiple subsequent artifacts demonstrates applicability across multiple contexts in the media domain. The adaptation achieved in this sociotechnical study, to our knowlegde, is a world first, extending presence theory in foreign media contexts by reconceptualizing immersion as a coconstructed experience.History: Jeffrey Parsons, Senior Editor; Samir Chatterjee, Associate Editor.