Script-Level Word Sample Augmentation for Few-Shot Handwritten Text Recognition

Abstract

The variety of handwriting styles and the scarcity of training data often result in poor performance of character recognizer. Rather than tedious data collection and annotation, researchers prefer to use low-cost data augmentation to improve the robustness of the recognizer. However, most existing data augmentation methods treat handwritten text as ordinary images and generate new samples through holistic transformation, which greatly limits the diversity of generated samples. To solve the problem, this paper proposes a script-level handwritten text augmentation method, where each component is treated as a Bézier curve. Specifically, we first segment the character into components based on skeleton detection. Then, we move the control points of each component according to the prior knowledge of languages. Finally, we transform the component by Bézier curves and assemble them into new samples. Our method is simple, controllable, and friendly to few-shot handwritten text. Experiments on four datasets in different languages show that the proposed script-level augmentation method performs better than the holistic augmentation methods. Apart from it, we also modify Affine transformation, a commonly used augmentation method, from a holistic to script-level way. Experimental results demonstrate that script-level affine can achieve better performance than holistic affine in the character recognition task. Our code is available at https://github.com/IMU-MachineLearningSXD/script-level_aug_ICFHR2022.