Rediscovering Automatic Detection of Stuttering and Its Subclasses through Machine Learning—The Impact of Changing Deep Model Architecture and Amount of Data in the Training Set

Abstract

Featured Application: The present investigation shows a methodology that can support a speech therapist by automatically classifying various types of speech disorders. This work deals with automatically detecting stuttering and its subclasses. An effective classification of stuttering along with its subclasses could find wide application in determining the severity of stuttering by speech therapists, preliminary patient diagnosis, and enabling communication with the previously mentioned voice assistants. The first part of this work provides an overview of examples of classical and deep learning methods used in automated stuttering classifications as well as databases and features used. Then, two classical algorithms (k-NN (k-nearest neighbor) and SVM (support vector machine) and several deep models (ConvLSTM; ResNetBiLstm; ResNet18; Wav2Vec2) are examined on the available stuttering dataset. The experiments investigate the influence of individual signal features such as Mel-Frequency Cepstral Coefficients (MFCCs), pitch-determining features in the signal, and various 2D speech representations on the classification results. The most successful algorithm, i.e., ResNet18, can classify speech disorders at the F1 measure of 0.93 for the general class. Additionally, deep learning shows superiority over a classical approach to stuttering disorder detection. However, due to insufficient data and the quality of the annotations, the results differ between stuttering subcategories. Observation of the impact of the number of dense layers, the amount of data in the training set, and the amount of data divided into the training and test sets on the effectiveness of stuttering event detection is provided for further use of this methodology.