Neural machine translation for harmonized system codes prediction

Abstract

The harmonized system codes (HS codes) are used worldwide to categorize products in international shipments. In its basic form HS codes come in 6 digit format, subdivided hierarchically into groups of two digits (chapters, headings and subheadings). When shipping products, it is mandatory to specify a HS code for the purpose of producing a custom declaration. Currently the process is mostly carried out by human experts who take a decision on the HS code to be assigned to a shipment depending on the item description provided by the shipper. As such the process is time consuming and prone to errors due to generic, incomplete or non-interpretable descriptions. The objective of this research is to automate the classification of HS codes in order to increase productivity to cope with extra volume in the custom classification area. For the purpose of testing the developed models, we used an anonymized data set of shipments provided by DHL. The main contribution of this paper is we applied a deep learning model which have not been tried on tackling the HS code classification problem: an attention-based neural machine translation (NMT) model with integration of hierarchical loss. The model can classify around 29% percentage of the dataset where the model's accuracy can reach 85%.