Federated learning for scam classification in small Indonesian language dataset: an initial study

Abstract

Most digital phishing or scam trick users into fraudulent links and is more effective against users with low technology literacy, like in Indonesia. Machine learning is widely used for scam classification, but most require sending the messages to a centralized server. This induces privacy concern as messages might contain private data. Federated learning (FL) was proposed to allow user devices to train models locally without sending data to server. In this work, we examined the use of FL with gated recurrent unit (GRU) model for classifying scam messages in Indonesian language with small dataset. We provided two FL-based baseline models (FedAvg and daisy-chained algorithms) and a dataset for scam classification in Indonesian language. We examined the models based on these performance metrics; precision, recall, F1, selectivity, and balanced accuracy. Despite the performance, we pointed out characteristics of the FL algorithms and the hyperparameters for this use case as pointers towards fine-tuning these baseline models. Overall, the FL model with FedAvg algorithm performed better in all metrics except recall.