Towards automatic question generation using pre-trained model in academic field for Bahasa Indonesia

Abstract

Exam evaluations are essential to assessing students’ knowledge and progress in a subject or course. To meet learning objectives and assess student performance, questions must be themed. Automatic Question Generation (AQG) is our novel approach to this problem. A comprehensive process for autonomously generating Bahasa Indonesia text questions is shown. This paper suggests using a decoder to generate text from deep learning models’ tokens. The suggested technique pre-processes Vectorized Corpus, Token IDs, and Features Tensor. The tensors are embedded to increase each token, and attention is masked to separate padding tokens from context-containing tokens. An encoder processes the encoded tokens and attention masks to create a contextual understanding memory that the decoder uses to generate text. Our work uses the Sequence-to-Sequence Learning architecture of BiGRU, BiLSTM, Transformer, BERT, BART, and GPT. Implementing these models optimizes computational resources while extensively exploring the research issue. The model uses context sentences as input and question sentences as output, incorporating linguistic elements like response placement, POS tags, answer masking, and named entities (NE) to improve comprehension and linguistic ability. Our approach includes two innovative models: IndoBERTFormer, which combines a BERT encoder with a Transformer decoder, and IndoBARTFormer, which decodes vectors like BERT. IndoTransGPT uses the Transformer as an encoder to improve understanding, extending the GPT model’s adaptability.