Visual-Linguistic Methods for Receipt Field Recognition

Abstract

Receipts are crucial for many businesses’ operation, where expenses are tracked meticulously. Receipt documents are often scanned into images, digitized and analyzed before the information is streamed into institutional financial applications. The precise extraction of expense data from receipt images is a difficult task owed to the high variance in fonts and layouts, the frailty of the print paper, unstructured scanning environments and an immeasurable amount of domains. We propose a method that combines visual and linguistic features for automatic information retrieval from receipt images using deep network architectures, which outperforms existing approaches. Our Skip-Rect Embedding (SRE) descriptor is demonstrated in two canonical applications for receipt information retrieval: field extraction and Optical Character Recognition (OCR) error enhancement.