User interfaces and 3D environment scanning for game-based training in mixed-reality spaces

Abstract

Game-based rehabilitation systems gain much interest recently due to fast advancement of natural human-machine interfaces including Augmented and Virtual Reality headsets, near-real time 3D body motion understanding and 3D environmental scanners. Game-based training and rehabilitation has quickly recognized the advantage of improving personal physical capabilities using games and competition as incentives for boosting patient’s compliance. Such systems call for new types of user interfaces, which seamlessly engage natural human senses and allow interaction as if one was in his/her natural environment. Furthermore, a possibility to exercise within a familiar home environment further improves the effectiveness of the rehabilitation. The core of the work presented here originates from the FP7-ICT-StrokeBack project and includes more recent advances in 3D scanning of large scale environments and introduces high precision 3D object modelling for realistic gaming environments from Horizon’2020-Reflective-SCAN4RECO project, both co-funded by the European Commission from FP7 and Horizon’2020 programs.