key concepts in elt

Multimodality
Matt Kessler

Downloaded from https://academic.oup.com/eltj/article/76/4/551/6645875 by Marina Gabelica on 02 April 2024

Over the past two decades, the concept of multimodality has garnered
considerable attention from language teachers and researchers both in
ELT contexts and beyond. The term multimodality refers to an individual’s
use of different modes (i.e. channels of communication) for the purpose of
conveying meaning. Such modes can consist of those that are linguistic,
visual, aural, gestural, or spatial in nature (Kress 2003). In the second
language (L2) learning literature, Tardy (2005) was one of the first to note
that many academic-related tasks were growing increasingly multimodal.
For instance, both teachers and their students are now frequently required
to construct slide-show presentations using various digital tools and
software (e.g. Google Slides, Microsoft PowerPoint). Importantly, such
tasks require individuals to employ multiple modes in order to convey
meaning, including written text (linguistic), images (visual), sound clips
(aural), hand gestures and facial expressions (gestural), and physical
elements arranged in a manner that facilitates comprehension (spatial).
Since Tardy’s observation, multimodality is increasingly regarded as an
essential component of ELT training, pedagogy, and education policy
(Grapin and Llosa 2020). In particular, due to the ubiquitous nature of
digital technologies, multimodality now permeates numerous facets of
teachers’ everyday practices. Many teachers may be unaware of the extent
to which they rely upon and leverage different modes as a part of their
pedagogic toolkit, using storyboard activities, picture description tasks,
flashcards, audio- and video-based files, multimedia projects, and other
multimodal materials.
As a concept, multimodality is generally regarded as emanating from
Halliday’s (1978) work involving social semiotics, which is concerned with
understanding the different modes of communication that humans use.
In his work, Halliday argued that researchers and instructors should go
beyond privileging sentence- and grammatical-level constructions. They
should not, in other words, solely explore, teach, and/or assess linguistic
resources. Instead, they should shift their attention to focusing on how
people use a combination of modes in different contexts for different
purposes. Illustrating this point, scholars such as Lim and Polio (2019)
have shown that academic fields often differ in the types of multimodal
tasks that their students are required to perform. For instance, students

© The Author(s) 2022. Published by Oxford University Press; all rights reserved.
 majoring in engineering and the sciences will likely need to create
assignments, such as lab reports, with the purpose of conveying meaning
while adhering to specific and defined academic genre conventions in
their field. Conversely, students in majors within the humanities may
produce assignments, such as portfolios. While these assignments also
must follow disciplinary conventions, students frequently may have more
creative license to integrate or mix various modes into their work. Thus,
given the sheer number and range of multimodal tasks in academia and
beyond, possessing multimodal literacy or ‘multiliteracies’ is considered
crucial (Jewitt 2008).

Downloaded from https://academic.oup.com/eltj/article/76/4/551/6645875 by Marina Gabelica on 02 April 2024

When it comes to investigating the impact of multimodality on L2
development, most research has tended to adopt qualitative, case study
designs (Li and Akoto 2021; Zhang, Akoto, and Li 2021). These studies
have often focused on students’ composition processes, or on teachers’
and students’ attitudes and beliefs about engaging in multimodal
assignments (e.g. Jiang and Luk 2016). While quantitative and mixed-
method studies are relatively scarce, there are a few studies that have
shown the potential of multimodality to facilitate L2 learning, especially
when compared to traditional (monomodal) composition. For instance,
Cho and Kim (2021) found no significant differences in measures
of quality, content, or language when comparing the outcomes of
groups who engaged in monomodal versus multimodal composition
tasks. Conversely, Vandommele et al. (2017) found that learners
who participated in collaborative multimodal writing assignments
outperformed a control group in terms of their texts’ perceived
communicative effectiveness, lexical diversity, text length, and other
measures.
Due to the paucity of quantitative research, there are current debates
about the extent to which language instructors should prioritize
multimodality in the classroom. Some researchers and teachers have
advocated for a ‘strong version’ of multimodality, stating that both
linguistic and non-linguistic modes should be given equal attention in
an effort to develop learners’ communicative competence (see Grapin
2019 for a discussion). Proponents of the strong version have pointed to
disciplines in the sciences as evidence that non-linguistic modes are of
equal value, since charts, figures, graphs, and tables are an integral part
of everyday communication (e.g. Lemke 1998). When it comes to ELT
purposes, strong-version proponents might argue that it is no longer
possible for teachers to devalue or to avoid non-linguistic modes in their
practice.
In contrast to the strong version, others (e.g. Qu, 2017) have advocated
for a ‘weak version’ of multimodality, in which non-linguistic modes
should only minimally support or supplement linguistic instruction. Such
supporters view non-linguistic modes as temporary scaffolds for learners
with limited L2 proficiency, which subsequently should be removed as
they progress. As Choi and Yi (2016) have noted, numerous studies have
shown that many ELT practitioners often share this view that linguistically
based forms are (and should be) the only legitimate form of literacy.
Proponents of a weak version have also pointed out that most forms of

Page 552 of 554 Matt Kessler

 standardized language testing (e.g. TOEFL, IELTS) overwhelmingly focus
on the linguistic mode alone.
While some continue to debate the prominence that non-linguistic modes
should receive in instruction, other scholars have questioned the utility
of multimodality altogether. For example, Manchón (2017) has voiced
concern that many multimodal activities may not be conducive to theories
of L2 acquisition such as interactionist approaches and sociocultural
theory, which stress input, interaction, and collaboration, among other
factors. In particular, Manchón has argued that if the multimodal activity
does not promote a real and authentic need to engage with the L2 and

Downloaded from https://academic.oup.com/eltj/article/76/4/551/6645875 by Marina Gabelica on 02 April 2024

to produce linguistic forms, then it may actually detract from or stifle
L2 development. However, more research is needed to investigate such
claims, including the relative effects of both the strong and weak versions
of multimodal instruction (see Lim and Kessler 2022 for a research
agenda).
Although lingering questions persist about the extent to which
multimodality should feature in the curriculum, in the interim, what is
clear is that discussions surrounding multimodality are likely to become
increasingly prevalent for ELT practitioners and researchers in the years
to come. Continued advancements in digital technologies have forever
changed what it means to be both a language teacher and a language
learner. As such, it is readily apparent that teachers must carefully
consider and be conscious of the ways in which they integrate multiple
modes into their practice.
Final version received May 2022

References Jiang, L. and J. Luk. 2016. ‘Multimodal Composing as

Cho, H. and Y. Kim. 2021. ‘Comparing the a Learning Activity in English Classrooms: Inquiring
Characteristics of EFL Students’ Multimodal into the Sources of its Motivational Capacity.’ System
Composing and Traditional Monomodal Writing: The 59: 1–11.
Case of a Reading-to-Write Task.’ Language Teaching Kress, G. 2003. Literacy in the New Media Age.
Research. London: Routledge.
Choi, J. and Y. Yi. 2016. ‘Teachers’ Integration Lemke, J. 1998. ‘Multiplying Meaning: Visual and
of Multimodality into Classroom Practices for Verbal Semiotics in Scientific Text’ in Reading
English Language Learners.’ TESOL Journal 7(2): Science., edited by J. Martin and R. Veel. Routledge.
304–27. Li, M., and M. Akoto. 2021. ‘Review of Recent
Grapin, S. 2019. ‘Multimodality in the New Content Research on L2 Digital Multimodal Composing.’
Standards Era: Implications for English Learners.’ International Journal of Computer-Assisted Language
TESOL Quarterly 53(1): 30–55. Learning and Teaching 11(3): 1–16.
Grapin, S. and L. Llosa. 2020. ‘Toward an Integrative Lim, J. and M. Kessler. 2022. ‘Directions for Future
Framework for Understanding Multimodal L2 Research on SLA, L2 Writing, and Multimodality’
Writing in the Content Areas.’ Journal of Second in The Routledge Handbook of Second Language
Language Writing 47. Acquisition and Writing, edited by R. M. Manchón and
Halliday, M. A. K. 1978. Language as Social Semiotic: C. Polio, 325–38. Routledge.
The Social Interpretation of Language and Meaning. Lim, J. and C. Polio. 2019. ‘Multimodal Assignments
London: Edward Arnold. in Higher Education: Implications for Multimodal
Jewitt, C. 2008. ‘Multimodality and Literacy in School Writing Tasks for L2 Writers.’ Journal of Second
Classrooms.’ Review of Research in Education 32(1): 241–67. Language Writing 47.

Multimodality Page 553 of 554

Manchón, R. M. 2017. ‘The Potential Impact of Beginning Learners of Dutch.’ Journal of Second
Multimodal Composition on Language Learning.’ Language Writing 36: 23–36.
Journal of Second Language Writing 38: 94–5. Zhang, M., M. Akoto, and M. Li. 2021. ‘Digital
Qu, W. 2017. ‘For L2 Writers, It Is Always the Multimodal Composing in Post-secondary L2
Problem of the Language.’ Journal of Second Language Settings: A Review of the Empirical Landscape.’
Writing 47: 92–3. Computer Assisted Language Learning.
Tardy, C. M. 2005. ‘Expressions of Disciplinarity and
Individuality in a Multimodal Genre.’ Computers and
Composition 22: 319–36. The author
Vandommele, G., K. Van den Branden, K. Van Gorp, Matt Kessler Department of World Languages,
and S. De Maeyer. 2017. ‘In-School and Out-of-School University of South Florida, Tampa, FL 33620, USA.

Downloaded from https://academic.oup.com/eltj/article/76/4/551/6645875 by Marina Gabelica on 02 April 2024

Multimodal Writing as an L2 Writing Resource for Email: mattjkess@gmail.com

Page 554 of 554 Matt Kessler