Penerapan Algoritma Transformer dalam Aplikasi Parafrase Teks Otomatis

Abstract

The development of Natural Language Processing (NLP) technology has enabled the creation of automated text manipulation applications, one of which is text paraphrasing. This study aims to implement a Transformer architecture with a focus on Indonesian text for automatic text paraphrasing applications. The model used is a pre-trained Text-to-Text Transfer Transformer (T5), which is fine-tuned using an Indonesian text corpus called the Indo-T5 model. During the training process, the model is trained to understand language structure and context in order to generate paraphrases that are not only grammatically correct but also semantically preserved. Evaluation was conducted using BLEU and ROUGE metrics to measure the similarity between the generated paraphrased texts and manual references. The evaluation results show that the model is capable of producing coherent, relevant paraphrased texts with a good level of lexical variation with a BLEU score of 50.1, and ROUGE-L of 61.7. Thus, this study demonstrates that Transformer-based models can be effectively applied to the task of text paraphrasing in Indonesian.