G3di: A gaming interaction dataset with a real time detection and evaluation framework

Abstract

This paper presents a new, realistic and challenging human interaction dataset for multiplayer gaming, containing synchronised colour, depth and skeleton data. In contrast to existing datasets where the interactions are scripted, G3Di was captured using a novel gamesourcing method so the movements are more realistic. Our detection framework decomposes interactions into the actions of each person to infer the interaction in real time. This modular approach is applicable to a virtual environment where the interaction between people occurs through a computer interface. We also propose an evaluation metric for real time applications, which assesses both the accuracy and latency of the interactions. Experimental results indicate higher complexity of the new dataset in comparison to existing gaming datasets.