Residual serialized cross grouping transformer for small scale sketch face recognition

Abstract

Sketch face recognition has recently gained significant attention in the field of computer vision due to its ability to quickly identify matched pairs of optical and sketch images. This technology has the potential to greatly improve the efficiency of law enforcement agencies in criminal investigations. However, there are still challenges that need to be addressed in sketch face recognition algorithms, such as modal differences and limited sample sizes. To overcome these issues, this study proposes a Residual Serialized Cross Grouping Transformer (RSCGT), which contains a residual serialized module to reduce the computation complexity, a two-layer Cross Grouping Transformer module that is capable of extracting modality-invariant context features, a domain adaptive module to mitigate the impact of modal differences. Additionally, we introduce a meta-learning training strategy to augment the generalization ability of this model. Experimental results demonstrate that the RSCGT achieves high accuracy in sketch face recognition tasks, even with small-scale datasets.