Indexicality and dynamic attention control in qualitative recognition of assembly actions

Abstract

Visual recognition of physical actions requires temporal segmentation and identification of action types. Action concepts are analyzed into attention, context, and change. Temporal segmentation is defined as a context switch detected by a switching of attention. Actions are identified by detecting “indexical” features which can be quickly calculated from visual features and directly point to action concepts. Validity of the indexicality depends on the attention and the context. These are maintained by three types of attention control: spatial, temporal and hierarchical. They are combined by a mechanism called “attention stack”, which extends at important points and winds up elsewhere. An action recognizer built upon the framework successfully recognized human assembly action sequences in real time and output qualitative descriptions of the tasks.