A feature-based knowledge distillation (FKD) for offline signature feature learning without signatures

Abstract

This paper proposes a novel approach to harnessing the knowledge of pre-trained expert models for training new Convolutional Neural Networks, particularly in domains with limited or unavailable task-specific data. The method is applied to offline handwritten signature verification (OffSV), a biometric field that faces challenges related to data scarcity, often due to regulatory constraints. The proposed Student-Teacher (S-T) framework employs feature-based knowledge distillation (FKD), integrating graph-based similarity for local activations and global similarity measures to guide the student model's training, using only handwritten text data. Notably, the models trained with this method show performance on par with, or exceeding, that of the teacher model across three widely used signature datasets. More importantly, these results are attained without employing any signatures during the feature extraction training process. This study demonstrates the efficacy of leveraging existing expert models to overcome data scarcity challenges in OffSV and potentially other related domains. The proposed methodology is available at https://github.com/dimTsourounis/FKD.