Research on Equipment and Algorithm of a Multimodal Perception Gameplay Virtual and Real Fusion Intelligent Experiment

Abstract

Featured Application: Authors are encouraged to provide a concise description of the specific application or a potential application of the work. This section is not mandatory. Chemistry experiments are an important part of chemistry learning, and the development and application of virtual experiments have greatly enriched experimental teaching. However, in the existing virtual experiments, there are problems such as low human–computer interaction efficiency, poor user sense of reality and operation, and a boring experimental process. Therefore, this paper designs a multimodal perception gameplay virtual and real fusion intelligence laboratory (GVRFL). GVRFL uses virtual and real fusion methods to interactively complete chemical experiments, which greatly improves the user’s sense of reality and operation. This method proposes a multimodal intention active understanding algorithm to improve the efficiency of human–computer interaction and user experience and proposes a novel game-based virtual–real fusion intelligent experimental mode that adds gameplay to the process of virtual–real fusion experiments. The experimental results show that this method improves the efficiency of human–computer interaction and reduces the user’s operating load. At the same time, the interaction between the real experimental equipment and the virtual experimental scene greatly improves the user’s sense of reality and operation. The introduction of game elements into the process of virtual and real fusion experiments stimulates students’ interest in and enthusiasm for learning.