Multimodality in Translation and Interpreting Studies:Theoretical and Methodological Perspectives

Abstract

Recent scholarly developments both within and outside translation studies attest to the growing perception among researchers that, in the pursuit of understanding processes of interlingual and intercultural transfer and mediation, analysing language is not enough. The centrality of (mostly written) language in translation studies research is hardly surprising, with linguistics being widely regarded as the discipline which has most informed the study of translation and interpreting since they emerged as a field of academic inquiry in the middle of the twentieth century (Baker 1996). The emphasis of early research on short, often decontextualised stretches of text (Baker and Pérez-González 2010) resulted in an excision of language – understood as text or discourse – from its context that has become the object of growing scrutiny by translation and interpreting scholars over the last two decades. More importantly, this displacement of language from context has favoured the analysis of language and its instantiation in discourse separately from other forms of meaning-making resources. This paper sets out to examine recent theoretical developments seeking to redress the displacement of language from other kinds of meaning-making resources and their impact on the theorisation of translation and interpreting.The starting premise of this chapter is that academic interest in non-verbal semiotic resources and their role in processes of interlingual and intercultural transfer is unevenly spread across different scholarly strands within the discipline. As far as the breadth of this research agenda is concerned, images appear to be the only non-linguistic meaning-making signs showing an increasingly recognised potential to inform research in translation studies. Dialogue interpreting, audiovisual and drama translation, to give but a few examples, still lack the theoretical and methodological concepts and tools to systematically analyse semiotic resources such as the gestures and facial expressions accentuating face-to-face conversation; the choices of fonts, colours and patterns of textual-visual interaction in printed advertisements; or the use of music and lighting in the staging of a drama production, respectively.This paper surveys ongoing research on how different semiotic resources shape translational behaviour in different communicative contexts, including but not limited to the interaction between speech and image in printed media and motion pictures; the modelling of composite semiotic systems, such as movement, gestures and gaze; the representation of identities and ideologies using non-verbal resources; and the conceptualisation of space, interpersonal perspective and salience in a range of settings, such as museums. I then move on to explore how insights imported from multimodal theory, as developed in the field of systemic functional linguistics and social semiotics, may help translation and interpreting scholars to gain new insights into old data. Key notions like ‘multimodal’, ‘multimedial’, ‘mode’, ‘modality’, ‘sub-mode’ and ‘medial realisation’ are introduced and explored in some detail.The contribution of multimodal insights to research in translation studies is also gauged in relation to new data and their contexts of production, as illustrated by the way in which different modes function semiotically when combined in the modern discourse worlds afforded by the computer and the Internet. In these ‘new media’, information is proliferating in forms which push our methods of sharing it effectively; the shape of discourse communities using, assessing and circulating translations is changing with the changing shape of texts; ideological currents engaging with the interpretation of translations are flowing beyond existing linguistic means of analysis and critique; and new amateur phenomena, mainly fandom and political activism, are increasingly appropriating translation and interpreting as a means to effect social change.The final section (before the conclusion) considers the methodological implications of multimodal research in translation and interpreting studies, with particular emphasis on new tools like multimodal transcriptions and multimodal corpora