Beyond watching: Action understanding by humans and implications for motion planning by interacting robots

Abstract

When you see an individual holding a baby in front of an elevator, you instinctively move to open the door for him. This seemingly obvious helping action is possible because you are able to immediately characterize, recognize and then understand his actions—that is, recognize the intention behind the observed individual’s actions, estimate the outcome of his current actions, predict his future actions and infer his constraints due to the baby in his hands. Fast action recognition and action understanding abilities make humans adept at social interactions, and are fundamental requirements for future robots in order for them to interact with humans. While other chapters in this book focus on action characterization and recognition through the use of dance notations, in this chapter we will focus on the problem of understanding recognized actions. In particular, we aim to elucidate how ideas from action understanding research in primates can help robots formulate behavior plans when they interact with humans and other robots. We first briefly review the historical concepts, and psychological and neuro-scientific findings on action understanding by primates. Next, we detail the possible computational mechanisms underlying action understanding by humans. We then highlight the controversies regarding these beliefs and explain the results of our recent study that answers some of these controversies. Finally, utilizing results from our study, we propose and explain a conceptual bio-mimetic framework for action understanding by robots, in order to enable them to plan helping and impeding behaviors during interactions, similar to humans.