Concepts in multimodal discourse analysis with examples from video conferencing
PDF

Keywords

human–computer interaction
language and interaction
mediation
multimodal discourse analysis
multimodal (inter)action analysis

How to Cite

Norris, S. . (2016). Concepts in multimodal discourse analysis with examples from video conferencing. Yearbook of the Poznan Linguistic Meeting, 2(1), 141–165. https://doi.org/10.1515/yplm-2016-0007

Abstract

This article presents theoretical concepts and methodological tools from multimodal (inter)action analysis that allow the reader to gain new insight into the study of discourse and interaction. The data for this article comes from a video ethnographic study (with emphasis on the video data) of 17 New Zealand families (inter)acting with family members via skype or facetime across the globe. In all, 84 social actors participated in the study, ranging in age from infant to 84 years old. The analysis part of the project, with data collected between December 2014 and December 2015, is ongoing. The data presented here was collected in December 2014 and has gone through various stages of analysis, ranging from general, intermediate to micro analysis.

Using the various methodological tools and emphasising the notion of mediation, the article demonstrates how a New Zealand participant first pays focused attention to his engagement in the research project. He then performs a semantic/pragmatic means, indicating a shift in his focused attention. Here, it is demonstrated that a new focus builds up incrementally: As the participant begins to focus on the skype (inter)action with his sister and nieces, modal density increases and he establishes an emotive closeness. At this point, the technology that mediates the interaction is only a mundane aspect, taken for granted by the participants.

https://doi.org/10.1515/yplm-2016-0007
PDF

References

Bateson, G. 1972. Steps to an ecology of mind. New York: Ballantine.

Bucholtz, M. and K. Hall. Forthcoming. “Embodied sociolinguistics”. In: Coupland, N. (ed.), Sociolinguistics: Theoretical debates. Cambridge: Cambridge University Press.

Geenen, J. 2013. Kitesurfing: Action, (inter)action and mediation. (PhD thesis, Auckland University of Technology.)

Geenen, J. Forthcoming. Multimodal acquisition of interactive aptitudes: A microgenetic case study.

Goffman, E. 1963. Behaviour in public places. New York: Free Press.

Goffman, E. 1974. Frame analysis. New York: Harper.

Gumperz, J. 1982. Discourse strategies. Cambridge: Cambridge University Press.

Hamilton, H. 1998. “Reported speech in survivor identity in on-line bone marrow transplantation narratives”. Journal of Sociolinguistics 2(1). 53–67.

Herring, S., D. Stein and T. Virtanen (eds). 2013. Pragmatics of computer-mediated communication. Berlin, New York: de Gruyter Mouton.

Kress, G. 2000. “Design and transformation: New theories of meaning”. In: Cope, B. and M. Kalantzis (eds.), Multiliteracies: Literacy learning and the design of social futures. Psychology Press. 153–203.

Kress, G. and T. Van Leeuwen. 1998. Reading images: A grammar of visual design. London: Routledge.

Kress, G. and T. Van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. London: Edward Arnold.

Ladefoged, P. 1975. A course in phonetics. Orlando: Harcourt Brace.

Latour, B. 2005. reassembling the social: an introduction to actor-network-theory. Oxford: Oxford University Press.

Makboon, B. 2015. Spiritual vegetarianism: Identity in everyday life of Thai non-traditional religious cult members. (PhD thesis, Auckland University of Technology.)

Norris, S. 2004. Analyzing multimodal interaction: A methodological framework. London: Routledge.

Norris, S. 2006. “Multiparty interaction: A multimodal perspective on relevance”. Discourse Studies 8(3). 401–421.

Norris, S. 2008. “Personal identity construction: A multimodal perspective”. In: Bhatia, V., J. Flowerdew and R.H. Jones (eds), New directions in discourse. London: Routledge. 132–149.

Norris, S. 2009. “Modal density and modal configurations: Multimodal actions”. In: Jewit, C. (ed.), Routledge handbook for multimodal discourse analysis. London: Routledge.

Norris, S. 2011a. Identity in (inter)action: Introducing multimodal (inter)action analysis. Berlin: de Gruyter Mouton.

Norris, S. 2011b. “Three hierarchical positions of deictic gesture in relation to spoken language: A multimodal interaction analysis”. Visual Communication 10(2). 1–19.

Norris, S. 2013a. “What is a mode? Smell, olfactory perception, and the notion of mode in multimodal mediated theory”. Multimodal Communication 2(2). 155–169.

Norris, S. 2013 b. “Multimodal (inter)action analysis: An integrative methodology”. In: Müller, C., E. Fricke, A. Cienki and D. McNeill (eds.), Body – language – communication. Berlin/New York: de Gruyter Mouton.

Norris, S. 2014. “The impact of literacy based schooling on learning a creative practice: Modal configurations, practices and discourses”. Multimodal Communication 3(2). 181–196.

Norris, S. (ed). 2015. Multimodality: Critical concepts in linguistics. (Vols. I–IV.) Abingdon: Routledge.

Norris, S. Forthcoming. Working with multimodal data: Research methods in multimodal discourse analysis. Hoboken, NJ: John Wiley and Sons.

Norris, S. and R.H. Jones. 2005. “Introducing mediated action”. In: Norris, S. and R.H. Jones (eds.), Discourse in action: Introducing Mediated Discourse Analysis. London, New York: Routledge. 17–19.

Pietikäinen, S., P. Lane, H. Salo and S. Laihiala-Kankainen. 2011. “Frozen actions in the Arctic linguistic landscape: A nexus analysis of language processes in visual space”. International Journal of Multilingualism 8(4). 277–298.

Pirini, J. 2015. Research into tutoring: Exploring agency and intersubjectivity. (PhD thesis, Auckland University of Technology.)

Pirini, J. 2016. “Intersubjectivity and materiality: A multimodal perspective”. Multimodal Communication 5(1). 1–14.

Schiffrin, D. 1987. Discourse markers. Cambridge: Cambridge University Press.

Schiffrin, D. 1994. Approaches to discourse. Oxford: Blackwell.

Scollon, R. 1997. “Handbills, tissues, and condoms: A site of engagement for the construction of identity in public discourse”. Journal of Sociolinguistics 1(1). 39–61.

Scollon, R. 1998. Mediated discourse as social interaction. London: Longman.

Scollon, R. 2001. Mediated discourse: The nexus of practice. London: Routledge.

Shohamy, E. and D. Gorter (eds.). 2008. Linguistic landscape: Expanding the scenery. London: Routledge.

Tannen, D. 1984. Conversational style: Analyzing talk among friends. Norwood, NJ: Ablex.

van Leeuwen, T. 1999. Speech, music, sound. London: Macmillan.

Van Lier, L. 1996. Interaction in the language curriculum: Awareness, autonomy and authenticity. London: Longman.

Wertsch, J. 1998. Voices of the mind: A sociocultural approach to mediated action. Cambridge, MA: Harvard University Press.

Wodak, R. 1989. Language, power and ideology: Studies in political discourse. Amsterdam, Philadelphia: John Benjamins.