esense 2.0: Modeling multi-agent biomimetic predation with multi-layered reinforcement learning

Abstract

Learning in multi-agent systems, especially with adversarial behavior being exhibited, is difficult and challenging. The learning within these complicated environments is often muddied by the multitudinous conflicting or poorly correlated data coming from the multiple agents and their diverse goals. This should not be compared against well-known flocking-type behaviors where each agent has the same policy; rather, in our scenario each agent may have their own policy, sets of behaviors, or overall group strategy. Most learning algorithms will observe the actions of the agents and inform their algorithm which seeks to form the models. When these actions are consistent a reasonable model can be formed; however, eSense was designed to work even when observing complicated and highly-interactive must-agent behavior. eSense provides a powerful yet simplistic reinforcement learning algorithm that employs model-based behavior across multiple learning layers. These independent layers split the learning objectives across multiple layers, avoiding the learning-confusion common in many multi-agent systems. We examine a multi-agent predator-prey biomimetic sensing environment that simulates such coordinated and adversarial behaviors across multiple goals. This work could also be applied to theater wide autonomous vehicle coordination, such as that of the hierarchical command and control of autonomous drones and ground vehicles.