LLM-Based Text Prediction and Question Answer Models for Aphasia Speech

Abstract

Aphasia, a brain injury-related linguistic problem, hinders communication. Current techniques generally struggle to handle aphasic speech's intricacies. BERT, short for Bidirectional Encoder Representations from Transformers, is a pre-trained natural language model that utilizes contextual information from both preceding and succeeding words in a sentence to predict the target word. This study uses BERT models to predict and fill in sentences for people with aphasia, using the AphasiaBank dataset. The patients' transcripts were thoroughly preprocessed, with nonverbal clues and redundant phrases removed. Because of the lack of control data, the accuracy of BERT in predicting masked tokens in aphasic speech was evaluated using a manual rating system with four raters. In addition, BERT was used for question-answering to increase context comprehension, underlining its ability to aid communication for those with aphasia. The preprocessing pipeline used advanced text-cleaning algorithms to ensure input data quality. The evaluation of BERT performance yielded satisfactory results with strong inter-rater reliability. The inter-rater correlation was remarkably strong, overall coefficients ranging from 0.61 to 0.74, suggesting a substantial level of agreement (Fleiss' Kappa Score: 0.32). BERT's predictions demonstrated a significant degree of contextual relevance and grammatical accuracy, as proven by ratings that were primarily above 3.0. The box plots also suggested a minimal number of outliers. The goal of this method is to improve the accuracy of speech prediction, which is beneficial for caregivers and speech therapists. BERT shows its nuanced capability in Aphasia sentence completion tests by exhibiting exceptional performance in terms of contextual appropriateness and grammatical correctness, as confirmed by manual evaluation.