An Interactive Virtual Home Navigation System Based on Home Ontology and Commonsense Reasoning

Abstract

In recent years, researchers from the fields of computer vision, language, graphics, and robotics have tackled Embodied AI research. Embodied AI can learn through interaction with the real world and virtual environments and can perform various tasks in virtual environments using virtual robots. However, many of these are one-way tasks in which the interaction is interrupted only by answering questions or requests to the user. In this research, we aim to develop a two-way interactive navigation system by introducing knowledge-based reasoning to Embodied AI research. Specifically, the system obtains guidance candidates that are difficult to identify with existing common-sense reasoning alone by reasoning with the constructed home ontology. Then, we develop a two-way interactive navigation system in which the virtual robot can guide the user to the location in the virtual home environment that the user needs while repeating multiple conversations with the user. We evaluated whether the proposed system was able to present appropriate guidance locations as candidates based on users’ speech input about their home environment. For the evaluation, we extracted the speech data from the corpus of daily conversation, the speech data created by the subject, and the correct answer data for each data and calculated the precision, recall, and F-value. As a result, the F-value was 0.47 for the evaluation data extracted from the daily conversation corpus, and the F-value was 0.49 for the evaluation data created by the subject.