Integration of physical devices into game-based virtual reality

Abstract

Virtual reality (VR) systems have the potential for alleviating the existing constraints on various natural and social resources. Currently, real-time applications of VR systems are hampered by the tediousness of creating virtual environments. Furthermore, today's VR systems only stimulate the human senses of vision, hearing -and to some extent touch -which prevents the system users from feeling fully immersed in the virtual environment. By integrating real physical devices with virtual environments, the user interactions with such systems can be improved and advanced technologies such as the MS Kinect system could be used to augment the environments themselves. While existing development platforms for VR systems are expensive, game engines provide a more efficient method for integrating VR with physical devices.In this paper, an efficient approach for integrating virtual environments and physical devices is presented. This approach employs modifications of games that are based on commercially available game engines for implementing the virtual environments in conjunction with the application of Dynamic Link Libraries (DLLs) for realizing versatile communications between these virtual environments and various application platforms, which in turn can interact with the physical devices outside of the virtual environments. This paper is divided into four sections. In the first section, the motivation for the developments described here is discussed, followed by a description of the method used to integrate virtual environments with physical devices in the second section. In the third section, an interactive and collaborative laboratory environment based on a multi-player computer game engine that is linked to physical experimental setups is presented as an example of a VR system. In the final section, some additional promising applications of the developed platform and the corresponding challenges are briefly introduced.