Application of Multimodal NLP Instruction Combined with Speech Recognition in Oral English Practice

Abstract

In order to study the application of multimodal NLP instruction combined with speech recognition based on hybrid deep learning in oral English practice, firstly, the basic principle of speech recognition technology is introduced. The concept of hidden Markov model and three key algorithms are explained, and its simulation and implementation in speech recognition application are realized. The architecture and key technologies of the system are introduced. Then, it introduces the specific application of deep learning in NLP. Finally, Chinese teachers with oral English teaching experience participate in the recording. The effective reading time of each person is 65 minutes, and the reading sentences are 3100 sentences. The total number of people is 80 (40 men and 40 women). The sentences cover 1595 spoken English words. Conduct oral English training. The experimental results show that the recognition accuracy decreases by about 2%, but the recognition speed increases by 10 times. In addition, the scoring accuracy is equivalent to that of the platform system. The accuracy of this method in instruction classification is increased, which verifies the feasibility and effectiveness of this method. In the future, attention mechanism will be used to expand this method.