Capturing reality for a billiards simulation

Abstract

A mixed reality platform for a billiards simulation was implemented. For a better training experience, the system uses a natural and tangible user interface as input. The system will capture the reality of user manipulations, including the ball motion, observer position, and gesture. The primary input sensor is an RGB-depth camera. With the help of depth information, the capturing process becomes more robust and efficient. In this physical mini-pool table, an inexperienced user can pick a cue and hit a ball easier to get practical training in a game of billiards. However, distance perception is very critical in training. The display simulating the motion parallax effect is helpful for the user to perceive the distance and orientation in a virtual simulation. For convenience, the system allows a natural gesture to interact with the system. With this system, we demonstrate that capturing reality bring real experiences into the virtual world.