Performance Analysis and Development of QnA Chatbot Model Using LSTM in Answering Questions

Abstract

This research aims to evaluate the performance of a Long Short-Term Memory (LSTM) based chatbot in answering questions (QnA). LSTM is a type of  Recurrent Neural Network (RNN) architecture specifically designed to overcome vanishing gradient problems and can store long-term information. The method used is 5-fold cross-validation to train the chatbot model with 15 epochs at each fold using the dataset provided. The results showed variations in model performance at each fold. At the 5th fold, there was a decrease in performance with 84.63% accuracy, 96.36% precision, 64.9% recall, and 69.84% loss value. This finding shows that there is variability in the performance of the QnA chatbot model at each fold. In conclusion, the LSTM chatbot model can provide good answers with high accuracy and precision. Still, performance variations need to be considered in the use of this chatbot.