Using hierarchical temporal memory for recognition of signed Polish words

Abstract

The paper is concerned with automatic vision-based recognition of hand gestures expressing isolated words of Polish Sign Language (PSL). The Hierarchical Temporal Memory (HTM) [3] is applied. This tool replicates the structural and algorithmic properties of the human neocortex. The gestures are spatio-temporal entities. Therefore we believe the HTM can be able to identify and use the gesture’s subunits (counterparts of phonemes) organized in spatio-temporal hierarchy. The paper discusses the preparation issues of the HTM and presents results of the recognition of 101 words used in everyday life at the doctors and in the post office.