Robot-centric activity recognition ‘in the wild’

Abstract

This paper considers the problem of recognizing spontaneous human activities from a robot’s perspective. We present a novel dataset, where data is collected by an autonomous mobile robot moving around in a building and recording the activities of people in the surroundings. Activities are not specified beforehand and humans are not prompted to perform them in any way. Instead, labels are determined on the basis of the recorded spontaneous activities. The classification of such activities presents a number of challenges, as the robot’s movement affects its perceptions. To address it, we propose a combined descriptor that, along with visual features, integrates information related to the robot’s actions. We show experimentally that such information is important for classifying natural activities with high accuracy. Along with initial results for future benchmarking, we also provide an analysis of the usefulness and importance of the various features for the activity recognition task.