License plate detection and recognition based on the YOLO detector and CRNN-12

Abstract

This paper focuses on the detection and recognition of Chinese car license plate in complex background. Inspired by the success of Deep Convolutional Neural Network (DCNN) and Recurrent Neural Network (RNN) in the field of object detection and image recognition, we propose to apply the YOLO detector for license plate detection, and Convolutional Recurrent Neural Network (CRNN) for recognition, which achieves state-of-the-art recognition accuracy. Firstly, we trained YOLOv2 and YOLOv3 for license plate detection, and compared their detection performance. Secondly, we designed and trained a network, named as CRNN-12, for license plate recognition. CRNN-12 contains a DCNN which is used to extract features and a 2-layer bidirectional Gated Recurrent Unit (GRU) which is used to decode the feature sequences. Connectionist Temporal Classification (CTC) loss function is used for the purpose of jointly training DCNN and RNN. The benefits of this approach are as follows: (1) It realizes end-to-end recognition without segmentation; (2) GRU can make better use of contextual information of license plate images, which leads to improved recognition accuracy; (3) License plate with different number of characters can be recognized by one network.