Inter-Semiotic Translation within the Space of the Multimodal Text

Abstract

Though Jakobson conceptualized inter-semiotic transfer as a valid form of translation between texts some 40 years ago, it remained a relatively marginal area of investigation until recently. Additionally, we may note that information and communication technologies (ICTs) have also markedly modified previously held notions of what constitutes a “text”. Basing my research on these two observations, this paper will draw attention to inter-semiotic translation within the space the newscast, a prime example of an ever-evolving multi-modal text created by newer forms of ICTs. More specifically, the focus lies in how we can posit translational activity in the construction of the newscast (for instance, how the “visual” images translate the “verbal” narration of the newscaster or journalist). This conceptualization of “news-making” will lead us to consider the ways in which cultural stereotypes are created through the “translation” of interacting texts (verbal, visual, aural) on the same interface. To suggest that newscasts, and by extension media, proliferate cultural stereotypes, is by no means novel. However, to consider how inter-semiotic translation plays a role in their creation may be a departure from previous paradigms. In fact, Kress and van Leeuwan state: “This incessant process of ‘translation’, or ‘transcoding’ – ‘transduction’ – between a range of semiotic modes represents […] a better, more adequate understanding of representation and communication (2006:39)”. Furthermore, because cultural stereotyping is often at the root of conflict, this type of investigation becomes, we suggest, all the more worthwhile in understanding how we “translate” difference across borders, semiotic or otherwise.