Robosherlock: Unstructured information processing framework for robotic perception

Abstract

A pressing question when designing intelligent autonomous systems is how to integrate the various subsystems concerned with complementary tasks. Robotic vision must provide task relevant information about the environment and the objects in it to various planning related modules. In most implementations of the traditional Perception-Cognition-Action paradigm these tasks are treated as quasi-independent modules that function as black boxes for each other. Often these subsystems are running in completely different frameworks, with a thin communication interface or middle-ware between them. While each subproblem poses specific requirements that can make fusing them more challenging, perception can benefit tremendously from a tight collaboration with cognition. In the following, a common framework for cognitive perception, based on the principle of unstructured information management (UIM) will be presented, called RoboSherlock. UIM has proven itself to be a powerful paradigm for scaling intelligent information and question answering systems towards real-world complexity. Complexity in UIM is handled by identifying (or hypothesizing) pieces of structured information by applying ensembles of experts for annotating information pieces, and by testing and integrating these isolated annotations into a comprehensive interpretation. RoboSherlock is an open source software framework for unstructured information processing in robot perception that demonstrates the potential of the paradigm for real-world scene perception.