Gesture recognition performance score: A new metric to evaluate gesture recognition systems

Abstract

In spite of many choices available for gesture recognition algorithms, the selection of a proper algorithm for a specific application remains a difficult task. The available algorithms have different strengths and weaknesses making the matching between algorithms and applications complex. Accurate evaluation of the performance of a gesture recognition algorithm is a cumbersome task. Performance evaluation by recognition accuracy alone is not sufficient to predict its successful realworld implementation. We developed a novel Gesture Recognition Performance Score (GRPS) for ranking gesture recognition algorithms, and to predict the success of these algorithms in real-world scenarios. The GRPS is calculated by considering different attributes of the algorithm, the evaluation methodology adopted, and the quality of dataset used for testing. The GRPS calculation is illustrated and applied on a set of vision based hand/ arm gesture recognition algorithms reported in the last 15 years. Based on GRPS a ranking of hand gesture recognition algorithms is provided. The paper also presents an evaluation metric namely Gesture Dataset Score (GDS) to quantify the quality of gesture databases. The GRPS calculator and results are made publicly available (http://software.ihpc.a-star.edu.sg/grps/).