Detecting Gestures Through a Gesture-Based Interface to Teach Introductory Programming Concepts

Abstract

The goal of this research is to find an algorithm that is capable of recognizing gestures drawn in a visual and gesture-driven interface used to teach introductory programming concepts. Our system combines components from Google’s Blockly, a visual programming language with a drag-and-drop puzzle piece interface, and Microsoft’s Xbox Kinect which is used to perform skeletal tracking. We focus on two supervised machine learning clustering algorithms, centroid matching and medoid matching, to detect gestures.