An UE4 Plugin to Develop CVE Applications Leveraging Participant’s Full Body Tracking Data

Abstract

This technical paper describes a software framework that has been conceived in order to develop collaborative, immersive Virtual Reality applications with the distinctive ability to be fed by real-time full-body tracking data of the multiple collaborating participants. This feature has a key role both to improve the mutual awareness in the visually immersive CVE, and to exploit complete body poses for human-centered joint activities and related assessments. By following recent trends of VR research application development, the framework has been based on a popular game engine (Unreal Engine 4) and leverages its powerful built-in facilities for networked, multiplayer applications and for character skinning. The paper illustrates the implemented techniques for real-time size-independent mannequin animation that could be usefully applied for the same task to different game engines or VR development environments, and also describes the data management strategies devised for an efficient implementation of a CVE architecture dealing with full-body tracking data flows. The knowledge gained during this engine-specific work will be useful for the future definition of an engine-independent communication framework for CVEs among single-user VR applications, based on heterogeneous game engine technologies, that is a strongly-felt need for promoting collaboration among VR research groups and in multi-partner industrial projects.