Decoding Gestural Iconicity

Abstract

Speakers frequently perform representational gestures to depict concepts in an iconic fashion. For example, a speaker may hold her index finger and thumb apart to indicate the size of a matchstick. However, the process by which a physical handshape is mentally transformed into abstract spatial information is not well understood. We present a series of experiments that investigate how people decode the physical form of an articulator to derive imaginary geometrical constructs, which we call “gesture form.” We provide quantitative evidence for several key properties that play a role in this process. First, “profiling,” the ability to focus on a structural subunit within the complex form of the physical hand. Second, “perspective,” for which we show that one and the same handshape seen from different perspectives can lead to different spatial interpretations. Third, “selectivity,” the fact that gestures focus on specific spatial features at the expense of others. Our results provide a first step toward mapping out the process of how representational gestures make the communication of spatial information possible.