Dynamic contextualization and comparison as the basis of biologically inspired action understanding

Abstract

People exhibit a robust ability to understand the actions of others around them. In this work, we identify two biologically inspired mechanisms that we hypothesize to be central in the function of action understanding. The first module is a contextual predictor of the observed action, given the goal-directed movement towards objects, and the actions that are allowed to be performed on the object. The second module is a kinematic trajectory parser that validates the previous prediction against a set of learned templates. We model both mechanisms and link them to the environment using the cognitive framework of Dynamic Field Theory and present our first steps into integrating the aforementioned modules into a consistent framework for the purpose of action understanding. The two modules and the combined architecture as a whole are experimentally validated using a recording of an actor performing a series of intentional actions testing the ability of the architecture to understand context and parse actions dynamically. Our initial qualitative results show that action understanding benefits from the combination of the two modules, while any module alone would be insufficient to resolve ambiguity in the perceived actions.