Multimodal Literacy

First created as part of the Digital Humanities Praxis course in the spring of 2012 at the CUNY Graduate Center, Tandem explores the generation of datasets comprised of text and image data by leveraging Optical Character Recognition (OCR), Natural Language Processing (NLP) and Computer Vision (CV). This project builds upon that earlier work in a new programming framework. While other developers and digital humanities scholars have created similar tools specifically geared toward NLP (e.g. Voyant-Tools), as well as algorithms for image processing and feature extraction on the CV side, Tandem explores the process of developing a more robust and user-friendly web-based multimodal data generator using modern development processes with the intention of expanding the use of the tool among interested academics. Tandem functions as a full-stack JavaScript in-browser web application that allows a user to login, upload a corpus of image files for OCR, NLP, and CV based image processing to facilitate data generation. The corpora intended for this tool includes picture books, comics, and other types of image and text based manuscripts and is discussed in detail. Once images are processed, the application provides some key initial insights and data lightly visualized in a dashboard view for the user. As a research question, this project explores the viability of full-stack JavaScript application development for academic end products by looking at a variety of courses and literature that inspired the work alongside the documented process of development of the application and proposed future enhancements for the tool. For those interested in further research or development, the full codebase for this project is available for download.

This paper uses a case study of the drawings, early writings and imaginative role play of two children to illustrate how children use a variety of modes to make meaning in ways that are creative and beyond the design and expectation of adults. It aims to valorise the kinds of practices in which children routinely engage but which are often overlooked and devalued by adults, both parents and teachers. Framed by social semiotic theories of communication, multimodal pedagogies and cognitive accounts of children's drawings, it illustrates how the children in this study work easily and seamlessly across a variety of materials and modes, using the semiotic resources available in their environments, to create imaginary worlds and express meanings according to their interests. In profiling these children, this paper lends support to the claim of multimodal pedagogies that it is the shifting across modes, as well as the freedom to choose the mode of expression, that engages the child's affect and creativity and builds agency and voice.

1) Le DVP (Document virtuel personnalisable) – définitions, objectifs et problèmes.
2) Deux exemples : la mise en ligne de manifestations scientifiques et la description de documentaires audiovisuels.
3) La sémiotique textuelle.
4) Le scénario sémiotique.

Historically, comics have been viewed as a "debased or simplified word-based literacy," explains Dale Jacobs, who considers comics to be complex, multimodal texts. Examining Ted Naifeh's Polly and the Pirates, Jacobs shows how comics can engage students in multiple literacies, furthering meaning-making practices in the classroom and beyond. We are better able to "hear" the narrator's voice because we can see what words are emphasized by the bold lettering, and we associate particular kinds of voices with the narrative voice of a pirate's tale, especially emphasized here by the shape of the text boxes.

This graduate project examined the stories of Katzie community members in order to bring forward intergenerational wisdom and practices connected to their lands and waters that are specific to Katzie people in an education context. My research process included interviewing 11 participants. Research grew out of five protocols of Coast Salish people I learned through my upbringing: starting with the local; acknowledging and expressing gratitude for place and people; getting an education; listening to one’s elders; and, loving one another. Findings from this study include four story themes and recommendations from participants’ interviews, ten methodological principles, and six pedagogical practices, that guide an understanding of Katzie literacy and pedagogy. Teachings from these stories compose a land-based pedagogy, including the sky and water, grounded in important relationships to the territorial places of Katzie people. Listening to the words of Katzie people is a first step toward reconciliation and restitution in local education systems.

Using participatory action research, the first researcher functioned as co-teacher in a fifth-grade class in a large northeastern city public school. The researcher and classroom teacher guided 28 students working in book clubs to compose digital stories in response to historical fiction. The research questions were as follows: (a) What interpretations did students have of their historical fiction novels through the mediational tools of digital storytelling? (b) How did the dynamics of the book club structure contribute to the students' interpretive work? Data sources included students' process and product work, video and audio recordings of work sessions, reflective notes and journal, a semi-structured interview with the teacher, and stimulated recall interviews with three case study book clubs. Both researchers used multimodal analysis, particularly the concept transmediation, concepts of interpretation in reader response, and grounded theory, informed by activity theory, to analyze data. Findings show students' expression of and limits to interpretation in the multimodal ensembles of their digital stories. The purposeful use of digital technology generated ongoing problem solving. Activity systems expanded students' learning by generating collaborative zones of proximal development, a dialectic among mediational tools, and opportunities to take on roles that shaped students' identities and repositioned who they could be in this learning community. The study shows the value of project-based multimodal responses using digital technologies in collaborative groups to develop students' comprehension of literary texts. The study

In this article we highlight analyses conducted in two qualitative literacy studies to discuss various implications of a blended, or hybrid, approach to multimodal analysis. By investigating several prominent frameworks commonly used together for the purpose of analyzing multimodal data, and describing our own experiences blending these frameworks, we determine that a hybrid approach is not necessarily ineffective at producing data interpretations, but that it is insufficiently reflexive of the role researcher positionality plays in multimodal analysis. We conclude the article by offering recommendations for supplementing hybrid analytical approaches through data co-construction and increased attention to researcher positionality.

Patent claims define the protection scope of the intellectual property sought by the patent applicant or patentee. Broad claims are valuable as they can describe more expansive rights to the invention. Therefore, if these claims are too broad a potential infringer will more easily argue against them. But if the claims are too narrow the scope of protection of the intellectual property is greatly reduced. Patent claims have to be, on the one hand, determinate and precise enough and, on the other hand, as inclusive as possible. Therefore patent applicants must find a balance in the broadness of the scope defined by their claims. This balance can be achieved by the choice of words with a convenient degree of semantic indeterminacy, by the choice of modifiers or other strategies. In fact, vagueness in patent claims is a desirable characteristic for such documents. A quantitative and qualitative analysis of a corpus of 350 U.S. patents provides a promising starting point to understand the linguistic instruments used to achieve the balance between property claim scope and precision of property description. To conclude, some issues relating vagueness and pragmatics are suggested as a line of further research.

The last two decades have seen an increasing media convergence and the promotion and distribution of popular narratives through cross-media texts. The superhero genre forms an important part of children's media environments and contemporary children's culture, in particular for boys. Children are not merely consumers of media texts, they are also actively engaged in a range of activities -fantasies, make-believe play, drawing, writing and other forms of meaning-making -reflecting, expanding and commenting on these media texts. Children as storytellers, players and artists draw upon familiar elements from superhero narratives, to create their own meanings. Children's ability to move across media platforms and across modes of meaning-making with particular ease, may explain the phenomenal success of cross-media narratives such as superhero stories. The paper provides an overview into different areas of children's participation in the superhero narratives within and without the context of formal educational settings and provides a longitudinal case study of one boy's engagement with superheroes in his play and meaning-making activities. The paper calls for a re-evaluation of children's media culture and cultural practices, including educational practices around superheroes.

This study draws on pre- and post-surveys of 574 upper primary students who engaged in
intensive work over two school terms to produce a multimodal narrative using desktop
animation software. It was thought that factors such as general ability with computers,
interest in composing stories and persistence with long-term projects might be influences of
successful completion or higher quality products. Over the course of the project, there is
evidence of student growth in the ICT-related skills as well as enjoyment and capacity with
story composition. However, concerns remain over the pedagogy of extended project work
and the impact of this on student learning in the affective domain. The impact of these
findings for classroom teaching and the planning of long-term ICT projects are discussed.

The contributors to this edited volume examine the simultaneous implementation of critical and digital literacies and explore ramifications for the development and assessment of critical digital literacies (CDL) curricula across educational contexts. We ask: How has the increasing ubiquity of digital literacies in and out of school affected our definitions of critical literacies? And how have our ever-changing perceptions of critical literacies affected how we define, teach, and engage in digital literacies? We believe that there is crucial work to be done at these intersections, work that builds upon the extensive bodies of critical and digital literacies research. Some issues and questions that chapters address are:

Malala Yousafzai is a Pakistani female education activist and the youngest-ever Nobel Peace Prize winner. This paper discusses language variation in two texts written by Malala, namely a blog (“Diary of a Pakistani Schoolgirl”, 2009), and a memoir (I am Malala, 2013) co-authored with the British journalist Christina Lamb. We assume that these two texts can be analysed with reference to some generic characteristics, namely, 1) spontaneity, interactivity, low self-monitoring in the blog and 2) planning, high control and self-monitoring in the memoir. These textual characteristics can be loosely associated to genres typical of speech and writing, respectively, hence a selective comparison of passages taken from the two texts will test these assumptions. A quantitative and qualitative analysis will be presented to this end. In the concluding remarks, the findings of this study will be discussed.

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas.<br> <br> a-mode.eumade4ll.eu<br> <br> <br> Keynote speakers:<br> <br> Marina Bondi<br> Carey Jewitt<br> Rodney Jones<br> Gunther Kress and Jeff Bezemer<br> David Machin<br> Theo van Leeuwen <br> A-MODE ORGANIZING COMMITTEE<br> <br> Elisabetta Adami – University of Leeds, UK<br> Cristina Arizzi – University of Messina, Italy<br> Styliani Karatza – National and Kapodistrian University of Athens,<br> Greece, and University of Leeds, UK<br> Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany<br> Ilaria Moschini – University of Florence, Italy<br> Sandra Petroni – University of Rome 'Tor Verg...

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas.<br> <br> a-mode.eumade4ll.eu<br> <br> <br> Keynote speakers:<br> <br> Marina Bondi<br> Carey Jewitt<br> Rodney Jones<br> Gunther Kress and Jeff Bezemer<br> David Machin<br> Theo van Leeuwen <br> A-MODE ORGANIZING COMMITTEE<br> <br> Elisabetta Adami – University of Leeds, UK<br> Cristina Arizzi – University of Messina, Italy<br> Styliani Karatza – National and Kapodistrian University of Athens,<br> Greece, and University of Leeds, UK<br> Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany<br> Ilaria Moschini – University of Florence, Italy<br> Sandra Petroni – University of Rome 'Tor Verg...

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas. Rome, Italy University of “Tor Vergata” 20 – 22 June 2019 a-mode.eumade4ll.eu _________________________________________________ Keynote speakers: Marina Bondi Carey Jewitt Rodney Jones Gunther Kress and Jeff Bezemer David Machin Theo van Leeuwen _________________________________________________ A-MODE ORGANIZING COMMITTEE: Elisabetta Adami – University of Leeds, UK Cristina Arizzi – University of Messina, Italy Styliani Karatza – National and Kapodistrian University of Athens, Greece, and University of Leeds, UK Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany Ilaria Moschini – University of Florence, Italy Sandra Petroni – University of Rome ‘Tor Vergata’, Italy Marc Rocca – Rocca Creative Thinking Limited, UK ...

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas.<br> <br> a-mode.eumade4ll.eu<br> <br> <br> Keynote speakers:<br> <br> Marina Bondi<br> Carey Jewitt<br> Rodney Jones<br> Gunther Kress and Jeff Bezemer<br> David Machin<br> Theo van Leeuwen <br> A-MODE ORGANIZING COMMITTEE<br> <br> Elisabetta Adami – University of Leeds, UK<br> Cristina Arizzi – University of Messina, Italy<br> Styliani Karatza – National and Kapodistrian University of Athens,<br> Greece, and University of Leeds, UK<br> Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany<br> Ilaria Moschini – University of Florence, Italy<br> Sandra Petroni – University of Rome 'Tor Verg...

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas.<br> <br> a-mode.eumade4ll.eu<br> <br> <br> Keynote speakers:<br> <br> Marina Bondi<br> Carey Jewitt<br> Rodney Jones<br> Gunther Kress and Jeff Bezemer<br> David Machin<br> Theo van Leeuwen <br> A-MODE ORGANIZING COMMITTEE<br> <br> Elisabetta Adami – University of Leeds, UK<br> Cristina Arizzi – University of Messina, Italy<br> Styliani Karatza – National and Kapodistrian University of Athens,<br> Greece, and University of Leeds, UK<br> Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany<br> Ilaria Moschini – University of Florence, Italy<br> Sandra Petroni – University of Rome 'Tor Verg...

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas.<br> <br> a-mode.eumade4ll.eu<br> <br> <br> Keynote speakers:<br> <br> Marina Bondi<br> Carey Jewitt<br> Rodney Jones<br> Gunther Kress and Jeff Bezemer<br> David Machin<br> Theo van Leeuwen <br> A-MODE ORGANIZING COMMITTEE<br> <br> Elisabetta Adami – University of Leeds, UK<br> Cristina Arizzi – University of Messina, Italy<br> Styliani Karatza – National and Kapodistrian University of Athens,<br> Greece, and University of Leeds, UK<br> Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany<br> Ilaria Moschini – University of Florence, Italy<br> Sandra Petroni – University of Rome 'Tor Verg...

The proposed study is an attempt to respond to the demands of todays' EFL teaching which should start to move away from focusing on the teaching of language skills only to preparing learners for different literacy practices they might encounter in their future involving visual, gestural, audio, spatial, digital dimension of communication. Although several study have been conducted in exploring the use of digital storytelling project to foster students' multimodal literacies, in Indonesian context, almost none of the researches focused on the development of students' multimodal literacies in orchestrating different use of all semiotic modes in making meaning and communicating ideas. This study will explore how the use of pedagogy of multiliteracies in the Digital Storytelling (DST) project can contribute to the students' multiliteracies. Utilizing a case study design, this study will involve 30 elementary school students from the fifth grade and 1 elementary school English teachers. The school site and the teacher is purposively chosen as they have previously joined a digital story project. Multiple method of data collection will be used to triangulate findings, among others are the result of observation, interviews with students and teachers, and the document analysis including students' story scripts and final product of DST. The analysis of the findings will be based on the description of the course and the project, procedure and instructional steps using the pedagogy of multiliteracies, students' use of multimodality in their digital story and students' knowledge and discovery on multimodality

Multiliteracies can support student open access and communal success in the context of multimodaling and multilanguaging and ultimately in the process foster self-directed learning. From academic literature the concerns regarding literacy levels are clear and the overemphasis on reading and writing (versus other skills) is evident. Furthermore, there is a need for an extension of multimodal learning in higher education. Multimodaling in terms of communication, learning/teaching and delivery must be accounted for and accommodated. A non-compartmentalised view of modes and modalities is necessary in educational contexts. In this regard, instructional technology can play an increasingly important role. The blending of approaches, delivery mediums or technologies will increasingly become the norm and lecturers and students should, therefore, be able to function effectively within the different modalities. Yet, a clear digital divide in South Africa poses problems to the effective implementation of multimodal learning. Support in terms of computer literacy and even computer anxiety must be interrogated in higher education.
When considering multimodal learning and multiliteracies, the multilingual nature of students at universities cannot be ignored. As such, a move towards multilanguaging by means of technology is proposed. There is ample evidence of successful integration of African languages in higher education and through technology multilanguaging can be achieved. In addition, language attitude planning as operationalisation of the language status planning process should be accommodated. In terms of access, the concept of open education is highly relevant in a South African context. Open educational resources can provide many possibilities in terms of both formal and epistemological access to a range of resources. Success in higher education can be reached through a communal constructivist approach where collaboration is facilitated through multimodal support. In addition, success would imply that students are self-directed and possess relevant multiliteracies. It is evident that the discussion on what literacy entails should be ongoing and be revised and repurposed as required during the learning, unlearning and relearning process.

This dissertation project examines patterns of stance in essays written by high- and low-performing students in two upper-level undergraduate courses, one in political theory and the other in economics. It employs methods of linguistic discourse analysis, drawing largely on Appraisal Theory (a subset of Systemic Functional Linguistics), in combination with methods from corpus linguistics and theoretical insights from rhetorical genre studies. It examines how recurring patterns of stance in students' essays correspond to the goals and assessment criteria for writing in the courses, as revealed through interviews with the instructors and analysis of selected course material. Through this robust set of analytic approaches, the study aims to make explicit patterns of stance in student writing that correlate with high- and low-graded essays and with the disciplinary contexts. The broader aim is to render explicit patterns of interpersonal meanings constructed in students' texts that construe such abstract qualities as critical reasoning, complexity and nuance in argumentation, and control of the discourse—features identified by the instructors as valued in student writing. The study contributes to the field of composition and rhetoric by pinpointing discursive resources that enable some student writers to construct more discipline-congruent styles of argumentation than others. Specific findings show that, while the two essay assignments require different ways of using language to construct valued stances, the high-performing writers in both contexts more consistently construct a "novice academic" stance while the low-performing writers more consistently construct a "student" stance. The former is marked by the rhetorical qualities of contrastiveness, dialogic control, critical distance, and discoursal alignment, or assimilation of the disciplinary discourse. In contrast, the “student” stance is marked by frequent personalizing moves, repeated references to the classroom discourse, and comparatively infrequent use of discursive resources that construe the rhetorical qualities listed above. These findings have implications for instruction in writing in the disciplines (WID) contexts, specifically in terms of how instructors can refine their metalanguage about writing for discussing stance with students explicitly and in detail.

Adolescence is regarded as a particularly malleable and difficult age in the development of social identity and conception of self and even more potentially problematic for multilingual and multiethnic English learners (ELs). In this chapter, we review scholarship from the new millennium including major shifts in theory and research involving identity development of adolescent ELs, specifically first-generation migrants to majority-English-speaking societies. Globalization, transnationalism, and the expansion of the digital world have all greatly impacted adolescent youth identity development and consequently have sparked new research in the field. In this review, we outline major strands of investigation of identity development in EL youth, including individual psychosocial processes, the significance of institutional and societal contexts, and the centrality of both face-to-face and multimodal interaction and semiotic practices. With discussions of multilingualism, multiculturalism, multimodality, and multiliteracies shaping current scholarship, the chapter concludes with directions for further research on the multifaceted and dynamic nature of EL adolescent identity development.

Literature abounds detailing the trauma experienced by Black and Latino males in traditional school environments, but less research has explored the role community-based youth programs play in specifically supporting Black and Latino males in counteracting, healing, growing, and thriving through the trauma experienced in educational contexts and beyond. In this qualitative research study examining the experience of eight Black and Latino male high school students attending programming at the Deep Center, a community-based afterschool program in the Deep South, participant observation data were triangulated with semi-structured interviews, analysis of creative artifacts (produced and performed), as well as organizational literature and materials.
Findings indicate that Black and Latino males are incredibly aware of the negative, racialized, and deficit-based labels that exist about them and how those labels determine how they are perceived, treated, and ultimately disciplined, a process first experienced in elementary school. Some Black and Latino males are directly antagonized, questioned, and called thugs or in a variety of ways--whether intentional or unintentional--publicly humiliated, shamed, or discriminated against by teachers. And when they lash out, punishment is harsh, swift, and uncompromising, without any consideration of school personnel as perpetrators. As a result, Black and Latino males silence themselves or completely disengage and give up in traditional school environments because they feel like no one cares about them. But in Deep programming, Black and Latino males felt empowered, affirmed, liked/loved, respected, and free to be vulnerable, resulting in the crafting of healing literacies, which I call “rainbows,” that use  autoethnography and performance to develop 1) a reflective art practice that empowers youth to describe their lived realities and engage in critical self-reflection, and 2) transformational relationships that create a safe and brave environment for vulnerability and critical caring and humanizing literacies, which, in turn, empower Black and Latino males to transform the conditions of their oppression through advocacy, activism, and art. The metaphor of “rainbows,” inspired by a poem written by one of the youth participants, is used as a framework for examining the key components of healing literacies, based on the scientific explanation of how rainbows are formed, through reflection, refraction, and dispersion.

This article argues for the importance of teaching multimodal literacy in schools so as to adequately equip students to navigate the multimodal communicative landscape. Developing multimodal literacy in students is about developing in them the ability to view multimodal texts critically and for them to represent their ideas through the production of effective multimodal texts. This article describes an instructional approach developed to teach multimodal texts and describes the trial of the approach in a secondary school in Singapore. The approach is informed by Systemic Functional Theory and is aligned to the Learning by Design Framework widely used in multiliteracies. The systemic functional approach provides scaffolds for students to access the meanings made in multimodal texts by introducing the features and typical functions of the text, as well as highlighting the common strategies used in these multimodal texts to make meaning. The changing nature of the communicative landscape wrought by new technologies is accentuating the need to broaden our definition of literacy. Kress (2003: 1) argues that literacy in the new media age will involve " human, cognitive/affective, cultural and bodily engagement with the world and on the forms and shapes of knowledge ". He proposes a shift away from an alphabetic literacy and to explore the new forms of literacy needed in today's world. To be considered literate in this day and age is to be able to effectively communicate multimodally. The " multimodal turn " (Jewitt 2009: 4) is a recognition that language is usually co-deployed with other semiotic resources and meaning is made multimodally as a result of the orchestration of these resources. Language has come to be understood " not as some discreetly independent entity, but rather as part of complex sets of interconnecting forms of human semiosis " (Christie 2002: 3). Communication, especially with multimedia and social media, involves not just language, but also the use of multimodal resources, such as images, videos, embodied action, and three-dimensional objects to make meanings in different contexts (Smith et al. 2014). As such, students need to develop complementary competencies not only in traditional domains, such as literacy and numeracy, which remain founda-tional, but also develop a fluency in multimodal literacy (Lim & Hung, 2016). In this light, the literacy curriculum today cannot remain as a language-focused curriculum. The literacy curriculum must evolve beyond just the teaching and learning of reading

In this paper [Multimodal literacy as learning process of interpersonal interaction: Decontextualized activities or meaningful multimodal actions?] I present the goals, the procedures and the latest results of a long-term action-research which is held on Primary schools. The main goal of this research is to investigate if meaningful “students' multimodal actions” contribute to the multimodal literacy development. The “students' multimodal actions” are organized based on students’ involvement to activities that drive their interest in all their stages, which are: the activity selection, the production using multiple modes of expression and communication, the collaborative revision, the presentation and the assessment of the results based on real communication situations. The main conclusions of this research are that multimodal literacy cannot be achieved from decontextualized activities, that are taught as isolated teaching subjects, but from students' involvement in “multimodal actions” which have a meaning to them and diffuse to all school subjects as well.

My combined interests in American literature and culture on the one hand, and digital culture, on the other, despite being so diverse as subjects, eventually blended in what I teach and in how I teach it, and became also the focus of my critical inquiry. Therefore, this paper is an outcome of the exploration of the intersections, transgressions and border crossings between the above two areas. Here I look for the possible answers to the question: What are the boundaries of traditional literary study in the digital era, and what are the benefits of the digital tools in the teaching and studying of literature in higher education?

Multimodality, a term first used in the late 1990s (Jewitt et al. 2016:2), is the study of how we make meaning by combining multiple signifying means or modes-for example, image with writing, music and body movement, speech with gesture-into an integrated whole. While more established disciplines such as linguistics, semiotics and musicology engage primarily with those semiotic resources that fall within their respective conceptual remits, multimodality investigates the synergies between co-occurring semiotic resources. From a multimodal standpoint, each mode "is understood as realizing different communicative work" (Jewitt 2014/2017:16) and is moulded by its own semiotic limitations, potentialities and affordances. Multimodal ensembles-whether they adopt the form of a graphic novel, a film, an illustrated textbook, a museum exhibit or an everyday conversational encounterresult from the interplay between the relevant cooperating modes. Multimodality specialists have "not as yet focused on questions of translation" (Taylor 2016:222), while translation scholars have been slow to engage with multimodal concepts and methods. Theoretical and methodological developments in translation studies until the 1990s were driven by the conceptualization of translation as a process of written language transfer where the printed word is the only signifying means at play. The study of spoken texts as loci of interpreting activity has similarly tended to revolve around their verbal fabric, often glossing over the semiotic contribution of the orality and corporeality of interpreter-mediated speech. This disciplinary emphasis on language-centred meaning-making processes largely derives from the influence of linguistics during the formative period of translation (and interpreting) studies (Baker 2005). It is also consistent with the entrenched prevalence of monomodality, understood as the dominance of one signifying constituent, such as written language, over other types of meaning-making resources. Approaches to multimodality Translation studies has not yet managed to articulate clearly how semiotics and multimodality relate to one another. Stecconi (2009:261) argues that, when used in a lax sense, semiotics encompasses "research that goes beyond verbal language", including "multimedia/multimodal material", a term he associates with Gottlieb's (2005) work. But Gottlieb's (2017:46) own contribution to this debate identifies multimodality, described as an addition to the more established body of scholarship on multi-channel texts and paraverbal translation, exclusively with the work of multimodality specialists (Baldry and Thibault 2006; Kress 2010). Leaving aside the profusion of terms that translation scholars use to designate their theoretical explorations of meaning-making resources and practices, they would appear to agree that multimodality falls within the wider remit of semiotics.

Multimodal digital communication is the main theme of this conference meant to attract multidisciplinary research on a wide range of issues from teaching and learning to analysing multimodal digital data appearing in multiple communication arenas. Rome, Italy University of “Tor Vergata” 20 – 22 June 2019 a-mode.eumade4ll.eu _________________________________________________ Keynote speakers: Marina Bondi Carey Jewitt Rodney Jones Gunther Kress and Jeff Bezemer David Machin Theo van Leeuwen _________________________________________________ A-MODE ORGANIZING COMMITTEE: Elisabetta Adami – University of Leeds, UK Cristina Arizzi – University of Messina, Italy Styliani Karatza – National and Kapodistrian University of Athens, Greece, and University of Leeds, UK Ivana Marenzi – L3S Research Center, Leibniz University of Hannover, Germany Ilaria Moschini – University of Florence, Italy Sandra Petroni – University of Rome ‘Tor Vergata’, Italy Marc Rocca – Rocca Creative Thinking Limited, UK ...

This chapter provides a framework – based on the theoretical background of self-directed multimodal demiurgic learning – that supports students to be able to act as content creators within higher education. This critical literature review shows how self-directed learning and multimodality provide affordances towards extending formal and epistemological access in order for students to take charge of their own learning and play a prominent role in generating new content. This chapter makes recommendations through a wider framework and description of a learning cycle towards enabling demiurgic access through self-directed multimodal learning within a South African university context.

Theorists have argued that the changed nature of contemporary texts to communicate meaning through multiple modes requires students to develop ‘multimodal’ literacy appropriate for engaging with these texts. This Professional Research Project investigates the effectiveness of explicitly teaching film metalanguage using a ‘word wall’ for promoting
the development of Year 7 students’ multimodal literacy. The study population comprised two Year 7 English classes with special learning needs (comprehension difficulties and EAL
students). The first six weeks of action research focused on explicitly teaching targeted vocabulary and students responding to multimodal texts; through a project-based learning pedagogy in the final two weeks, students composed and reflected upon their own multimodal text. Qualitative analysis of cloze activities, spelling tests, metalanguage use
tallies and student composing yielded an indirect existential phenomenology methodology: results focus on how the word wall strategy developed meaning vocabulary regarding the
higher-order metalanguage concepts ‘intertextuality’ and ‘salience.’ While the ‘word wall’ facilitated scaffolding discussion about metalanguage, higher-ability students were more able than mid-range students to accurately use the targeted metalanguage in both responding and composing. However, many students demonstrated understanding of multimodal concepts without using the targeted metalanguage. This raises implications for sequencing
learning to develop multimodal literacy, and appropriately assessing the literacy. It is recommended that future studies of developing multimodal literacy through metalanguage
investigate strategies for targeting specific special learning needs.

Over the last years new communication contexts have reshaped print-based literacy pedagogy into a multimodal one. The changing nature of literacy is also reflected in the new curriculum for early childhood education, however without clearly articulating how this shift will be implemented in educational settings. Consequently, teachers are not familiar with ways to promote students’ multimodal competencies and are not trained to develop instructional plans aligned with multimodal pedagogies. The article discusses an on-going exploratory research aimed to provide insight into teaching multimodal literacy in Early Childhood Education. Specifically, it focuses on formulating learning experiences in order to enhance preschoolers’ critical viewing competencies concerning the intentions and meaning strategies employed by commercial video producers. The research included a framework of teaching multimodal texts, such as commercial videos, based on previous frameworks found in literature reviews. Specific learning experiences with commercial videos were designed and implemented in a Greek Kindergarten with twelve students. These learning experiences were organized in several units. Each unit included the spontaneous engagement of students with various commercial videos, the guided analysis of the structural features and typical functions of the videos as well as the guided ‘reading’ of literal and inferential meaning in them based on the explicit teaching of both meaning strategies employed by video resources and a metalanguage for justified interpretations and detailed descriptions of students’ responses drawn from textual evidence. In the current article, this particular teaching framework is presented in detail as an integrated part of the research project – which is briefly discussed as well – for the implementation of multimodality in teaching practices in early childhood settings and the enhancement of preschoolers’ literacy skills. Also, the constraints and further issues of both the research project and the teaching framework are being discussed.

In addition to being praised for promoting the healing power and the pedagogical potential of literature, William Joyce and Moonbot Studios' The Fantastic Flying Books of Mr. Morris Less-more, a cross-media narrative,which consists of four thematically related but discrete versions of one story (a film, an e-book app, a picturebook and an IMAG-NO -TRON app), has been criticized for two main reasons. First, it has been revealed that the book emphasizes the significance of books at large at the expense of foregrounding the benefits and mechanics of the act of reading. Second, one of its components, the e-book app, has been regarded as ineffective in enhancing the reading experience due to its limited interactive options. Whereas we partially agree with the latter argument, we nevertheless argue that, if seen as a whole, the Lessmore combo employs a catalogue of metaphors, such as reading is sharing or reading is remaining young, with a view to deepening and refreshing the reader/viewer/user's appreciation of reading as an activity that fosters one's affective and cogni-tive development. Significantly, depending on the narrative platform, these metaphors can be cued textually, visually, musically or/and kinesthetically. Focusing specifically on the metaphor reading is engaging with fictive minds, this article explores how the Lessmore film and its remediations recycle that metaphor along with encouraging the experience of immersion or/and interactivity.

Une étude menée en collaboration avec des institutions muséales comme sites édu- catifs dans un milieu hautement plurilingue et multiculturel sert de toile de réflexion pour mieux comprendre comment de jeunes locuteurs plurilingues font sens de leurs langues et des pratiques d’écrit et mobilisent, en situation, des ressources plu- rilingues pour comprendre et apprendre. Nous présentons quelques réflexions à par- tir d’exemples tirés d’une étude pilote menée avec de jeunes enfants de 5 ans ayant participé à des ateliers de science organisés au musée. Au cours de ces ateliers, il leur a été demandé de documenter et d’illustrer leurs apprentissages au sujet des ani- maux et de leurs traces, au moyen du dessin et de la photo (Molinié, 2009 et 2014) et de la création collaborative d’un livre digital à l’aide d’une tablette tactile (Sandvik, Smørdal et Østerud, 2009). La contribution est l’occasion de poser quelques ques- tions sur le rôle du français dans le développement littératié au sein d’un environne- ment essentiellement anglophone où les langues chinoises sont largement dominantes par rapport à l’autre langue officielle du Canada.

A dissertation submitted in partial fulfilment of the requirements for the degree of Master of Education (Applied Linguistics)
This dissertation is dedicated to my children and co-researchers, Tia and Brandon.
This paper entails exploring creativity in everyday literacy practices that are undertaken within the home setting so that we can make better links between out of school contexts and school literacies. The motivation for this study is the socio-semiotic assumption that all signs are motivated (Eco, 1976 in Harste in Kabuto, 2014:163) and that creativity can be found in everyday activities of communication and sense-making (Tusting and Papen, 1970). By exploring young people's creativity in everyday literacy practices, we may acknowledge the importance of context by valuing both creativity within subjects as well as creativity in its more generalisable form.