Mixed reality for improving tele-rehabilitation practices

Abstract

In the present work we propose a methodology for improving tele-rehabilitation practices adopting mixed reality techniques. The implemented system analyzes the scene of a tele-rehabilitation practice acquired by a RGB-D optical sensor, and detects the objects present in the scene, identifying the type of object, its position and rotation. Furthermore we adopted Mixed Reality techniques to implement more complex rehabilitation exercises. After an initial training period, in which the set of objects are classified by the system, the method analyzes the acquired images in real time and the identified objects (which are included in the set of preliminarily identified objects) are evidenced with a rectangle, the size and location of which are variable. All regions of the image are analyzed and objects of different shape and size are identified. In addition in a file associated to the object, the most relevant object features are stored using XML format. The implemented method is able to identify a vast set of objects used regularly in tele-rehabilitation exercises and allows the therapist to perform the quantitative assessment of the patient practices.