Identification of Issues in Predicting Multi-Robot Performance through Model-Based Simulations

Abstract

Predicting the performance of intelligent multi-robot systems is advantageous because running physical experiments with teams of robots can be costly and time consuming. Controlling for every factor can be difficult in the presence of minor disparities (i.e. battery charge). Access to a variety of environmental configurations and hardware choices is prohibitive in many cases. With the eminent need for dependable robot controllers and algorithms, it is essential to understand when real robot performance can be accurately predicted. New prediction methods must account for the effects of digital and physical interaction between the robots that are more complex than just collision detection of 2D or physics-based 3D models. In this paper, we identify issues in predicting multi-robot performance and present examples of statistical and model-based simulation methods and their applicability to multi-robot systems. Even when sensor noise, latency and environmental configuration are modeled in some complexity, multi-robot systems interject interference and messaging latency, causing many prediction systems to fail to correlate to absolute or relative performance. We support this supposition by comparing results from 3D physics-based simulations to identical experiments with a physical robot team for a coverage task. [PUBLICATION ABSTRACT]