Animal-in-the-loop System with Multimodal Virtual Reality to Elicit Natural Olfactory Localization Behavior

Abstract

Herein, we proposed an “animal-in-the-loop” (AIL) system by introducing a robot and a virtual reality (VR) technology in a conventional insect behavior experiment. The setup provided sensory inputs to an insect, which mimic its natural environment and simultaneously measured the behavioral output. The proposed AIL system consisted of a multimodal VR device and a ground-running robot, both of which were connected wirelessly. The insect behavior was measured using a multimodal VR device, and the behavioral changes were transmitted to the robot as control inputs. Specifically, the multimodal VR device was equipped with three types of sensory stimulators, odor, wind, and vision, and each stimulator was controlled by the value of the corresponding sensor on the robot. The surrounding environment was observed using multiple sensors mounted on the robot, and the information was transmitted to the VR device to provide sensory stimuli to the insect. This system allowed the insect on the VR device to remotely control the robot and perform localization virtually. The localization trajectories of the proposed AIL system were similar to those of the free-walking experiment, and the tendency of the change in the heading angle during localization was also similar. Therefore, we found that using the AIL system enabled us to measure behavioral changes upon providing sensory stimuli to insects. These VR stimuli were similar to those encountered by the insect in free-walking experiments.