Multilabel-Thai Text Classification with Transformer-Rnn in Thai Banking Classification

Abstract

This research presents a novel approach to multilabel Thai text classification within the banking sector using a hybrid Transformer-RNN model. The study focuses on enhancing the accuracy and efficiency of document categorization in Thai language, addressing the complexities inherent to diverse banking documents. The model integrates BERT for pre-training to leverage contextual embeddings and incorporates an RNN to capture sequential dependencies in the text data. Evaluation of the model's performance was conducted using precision, recall, and F1 score metrics over a 10-fold cross-validation setup. The hybrid Transformer-RNN model demonstrated robust performance across multiple evaluation metrics. Specifically, it achieved an average precision of 0.823, recall of 0.817, and F1 score of 0.815. These results indicate the model's efficacy in accurately predicting multiple labels associated with various types of Thai banking documents. Comparative analysis against LSTM, CNN, BERT, and Transformer (Encoder-Only) models further validates the superiority of the proposed approach in handling complex multilabel classification tasks in the Thai banking domain. This research underscores the potential of hybrid Transformer-RNN models in advancing multilabel text classification capabilities, particularly in specialized domains like Thai banking. The findings highlight significant improvements in classification accuracy and model robustness, contributing to enhanced document management, regulatory compliance, and customer service within the banking industry. Future research directions could explore ensemble learning techniques, domain adaptation strategies, and the integration of domain-specific knowledge bases to further enhance the model's performance and applicability in real-world scenarios.