Multimodal reference to objects: An empirical approach

Abstract

In this chapter we report on an investigation into the principles underlying the choice of a particular referential expression to refer to an object located in a domain to which both participants in the dialogue have multimodal access. Our approach is based on the assumption that participants try to use as little effort as possible when referring to objects. This assumption is operationalized in two factors, namely the focus of attention and a particular choice of features to be included in a referential expression. We claim that both factors help in reducing effort needed to, on the one hand, refer to an object and, on the other hand, to identify it. As a result of the focus of attention the number of potential target objects (i.e., the object the speaker intends to refer to) is reduced. The choice of a specific type of feature determines the number of objects that have to be identified in order to be able to understand the referential expression. An empirical study was conducted in which pairs of participants cooperatively carried out a simple block-building task, and the results provided empirical evidence that supported the aforementioned claims. Especially the focus of attention turned out to play an important role in reducing the total effort.