Automatic Verbal Depiction of a Brick Assembly for a Robot Instructing Humans

Abstract

Verbal and nonverbal communication skills are essential for human-robot interaction, in particular when the agents are involved in a shared task. We address the specific situation where the robot is the only agent knowing about both the plan and the goal of the task, and has to instruct the human partners. The case study is a brick assembly. We here describe a multilayered verbal depictor whose semantic, syntactic, and lexical settings have been collected and evaluated via crowdsourcing. One crowdsourced experiment involves a robot-instructed pick-and-place task. We show that implicitly referring to achieved subgoals (stairs, pillars, etc) increases the performance of human partners.