Ancillary Diagrams: A Substitute for Text in Multimedia Resources?

Abstract

Multimedia resources conventionally convey their subject matter through a combination of descriptive and depictive representations. However, responsibility for explaining that content is typically skewed heavily towards multimedia’s descriptive components. This theoretical paper considers likely perceptual and cognitive processing requirements for internalizing these two sources of information during mental model construction. It uses the example of a multimedia resource consisting of written text and an accompanying overview picture to propose that much of the role usually allocated to text in such a resource could conceivably be reallocated to a set of ancillary diagrams. This proposal is based on an analysis suggesting that these diagrams are a better foundation for mental model building than is text. Consequently, replacing the text in a multimedia resource with appropriately designed ancillary diagrams should result in superior understanding. Likely benefits and costs of this approach as well as possibilities for its further development are discussed.