A handwriting recognition system based on visual input

Abstract

One of the most promising methods of interacting with small portable computing devices such as personal digital assistants is the use of handwriting. However, for data acquisition touch sensitive pads, which are limited in size, and special pens are required. In order to render this communication method more natural Munich & Perona [11] proposed to visually observe the writing process on ordinary paper and to automatically recover the pen trajectory from video image sequences. On the basis of this work we developed a complete handwriting recognition system based on visual input. In this paper we will describe the methods employed for pen tracking, feature extraction, and statistical handwriting recognition. The di erences compared to classical on-line recognition systems and the modi cations in the visual tracking process will be discussed. In order to demonstrate the feasibility of the proposed approach evaluation results on a small writer independent unconstrained handwriting recognition task will be presented.