Integrating LangChain and Large Language Models for Enhanced Dysarthria Speech Recognition and Communication

Abstract

Neurological dysarthria is a complex speech disorder, that poses major communication challenges and demands innovative computational solutions. A new approach to enhance dysarthria speech recognition is proposed in this research. Integrating LangChain's linguistic analysis capabilities with Large Language Models (LLMs) is a novel approach. This study proposes to enhance dysarthric speech recognition by introducing a novel Deep Learning-based End-to-End Dysarthria/Non-dysarthria Detection with Automatic Speech Recognition (DLEE-DNDDASR) to provide a robust and systematic solution for dysarthria speech recognition. First, the standard speech signals are converted into spectrograms, then the spectrograms are classified using a Deep Convolutional Neural Network (DCNN) based on the EfficientNet feature extractor with a SoftMax classifier. Further, LangChain, integrated with an LLM-based conversation model, determines the prior probability of word sequences from recognized speech signals and offers automatic responses to the queries of the dysarthric speaker. The proposed approach was evaluated with the TORGO database and showed an improvement in both detection and transcription accuracy. The average accuracy of 95.50% was achieved on an 80% TRAS and 20% TESS data split by the DLEE-DNDDASR technique. The results of the proposed framework consistently outperformed alternative approaches when compared to existing methodologies to improve dysarthric speech recognition and its communication outcomes. These findings focus on its potential for practical use in assistive technology and speech therapy and present a promising means of tackling these challenges in the real world.