Expiry-date recognition system using combination of deep neural networks for visually impaired

Abstract

Many drink packages have expiry dates written in dot matrix characters (digits and non-digits, e.g., slashes or dots). We collected images of these packages and trained two existing deep neural networks (DNNs) to combine and form a system for detecting and recognizing expiry dates on drink packages. One of the DNNs is an object-detection DNN and the other is a character-recognition DNN. The object-detection DNN alone can localize the characters written on a drink package but its recognition accuracy is not sufficient. The character-recognition DNN alone cannot localize characters but has good recognition accuracy. Because the system is a combination of these two DNNs, it improves the recognition accuracy. The object-detection DNN is first used to detect and recognize the expiry date by localizing and obtaining the size of the character. It then scans the expiry-date region and clips the image. The character-recognition DNN then recognizes the characters from the clipped images. Finally, the system uses both DNNs to obtain the most accurate recognition result based on the spacing of the digits. We conducted an experiment to recognize the expiry dates written on the drink package. The experimental results indicate that the recognition accuracy of the object-detection DNN alone was 90%, that of the character-recognition DNN alone was also 90%, and that combining the results of both DNNs was 97%.