Research on the Compilation of Multimodal Translation Teaching Materials in the New Media Era

: With the advent of the 21st-century digital era, multimodal information dissemination has increasingly become the predominant mode of communication. Multimodal translation plays an increasingly critical role in cross-cultural information transmission. However, the current pedagogical domain in translation primarily focuses on the cultivation of linguistic translation skills, with less attention to the training of students' multimodal translation competencies. To date, there are no teaching materials specifically designed for multimodal translation. In light of the significance of multimodal translation, this paper explores the compilation issues of multimodal translation teaching materials within the context of the new media era, aiming to guide and inspire the development of such resources.


Introduction
Entering the 21st century, digital, network, and new media technologies have been widely applied across various sectors, ushering in the era of "reading images."People's reading habits have shifted towards visuality, and multimodal texts that integrate verbal language, sound, images, and video have started to play a more significant role in information dissemination.Multimodal information dissemination has become the mainstream mode of communication in today's era.Within the field of translation, the objects and methods of translation have undergone revolutionary changes in response to these shifts.The objects of translation now extend well beyond the traditional scope of linear texts (Wang Shaoshuang, Qin Jianghua, 2018).Translators are often required to handle multimodal texts that blend verbal language, imagery, and sound, or to use various modal symbols to translate verbal texts.Engaging in multimodal translation, translators not only need to possess linguistic translation skills but also the ability to translate across different modalities.For instance, translators may need to integrate textual information with visual and auditory elements to maintain the integrity and coherence of the message.This undeniably poses new challenges and demands for the training of translation talent.Therefore, teaching multimodal translation should rightly become an essential component of translation education.
Despite the growing academic interest in multimodal translation, its importance in translation education is often overlooked.Current translation teaching still predominantly focuses on cultivating language translation skills, with insufficient systematic training in multimodal translation competencies.The lack of specialized teaching materials for multimodal translation poses an obstacle to training translators capable of meeting modern societal needs.Based on the characteristics of multimodal translation, this paper discusses the issues related to the compilation of multimodal translation teaching materials, offering guidance for their development and supporting the training of multimodal translation talent.

Multimodal Translation in the New Media Era
Modality, as defined by Kress (2009), is a "socially shaped and culturally given resource for making meaning."It encompasses "channels and mediums of communication, including symbol systems such as language, technology, images, colors, and music" (Zhu Yongsheng, 2007).Multimodality refers to the "use of multiple semiotic modes in the design of semiotic products and events" (Kress & Van Leeuwen, 2001:20).Multimodal translation involves the employment of non-linguistic sign modalities such as images, sounds, and colors, in conjunction with linguistic signs, to coparticipate in the recreation and generation of the original text's meaning (Wu Yun, 2022).
It is widely recognized that linguistic signs are not the sole carriers for the construction of meaning and the dissemination of information.Non-linguistic resources such as images, sounds, colors, and animations can also serve as effective carriers of meaning, facilitating communication and information dissemination.Different modal signs have their own strengths and limitations in constructing meaning.For instance, images are noted for their intuitiveness and specificity, while text is renowned for its abstractness and capacity for association, playing a unique role in elucidating the profound implications of phenomena and ideas (Zhou Xian, 2005).Therefore, the combination of various modal signs often allows for the complementary advantages of each modality, presenting richer information and offering readers a more vivid and direct reading experience.Furthermore, the development of new media technologies today greatly facilitates the integration of various modalities, promoting the prevalence of multimodal information dissemination.
Translation is a form of information dissemination, and the popularity of multimodal information dissemination has naturally led to the prevalence of multimodal translation.On one hand, an increasing number of multimodal texts are becoming the subjects of translation, necessitating the consideration of non-linguistic modal signs alongside linguistic ones.On the other hand, given that multimodal texts have become the mainstream medium for information dissemination, translators can also utilize the advantages of multimodal transmission when translating traditional, singlemodality linguistic texts to provide richer reading experiences and enhance cross-cultural communication effects.Recent years have seen a growing practice of using multimodal means for translating linguistic texts, which has been wellreceived, such as Tsai Chih Chung's comic series on traditional Chinese culture and the illustrated English translation of the "Classic of Mountains and Seas" by Ge Haowen (2021).
In the current era, international communication is undergoing a new developmental phase characterized by the accelerated integration of traditional and modern emerging media.Within this context, conducting international communication through multimodal forms has become a primary channel and method for achieving communication objectives (Wu Yun, 2022).Accordingly, the translation market's demand for translation talent is inevitably shifting towards those proficient in multimodal translation, who will possess a competitive edge in the future.Therefore, cultivating multimodal translation talent based on the core characteristics of current multimodal translation practices and new requirements for translators should become a focus in the training of translation talent.The development of translation teaching materials that highlight the features of multimodal translation is essential to support the training of such talent.

Compilation of Multimodal Translation Teaching Materials
In any educational system, the development of teaching materials is a crucial component, the importance of which cannot be overstated.High-quality teaching materials systematically provide essential knowledge and information, helping students build a solid foundation in their subject while guiding them towards deeper learning.Well-designed textbooks not only contain rich content but also consider the progression of knowledge, ensuring that the learning process is both logically clear and well-structured.The quality of textbook content directly affects teaching effectiveness.A good set of textbooks should accurately convey specialized knowledge while staying in sync with the latest academic research and practical applications.To a large extent, textbooks influence the quality of teaching and, consequently, the achievement of educational goals in talent cultivation.
In recent years, as Translation Studies has become an independent academic discipline and with the establishment of undergraduate majors and Master's programs in Translation and Interpreting (MTI), a plethora of translation textbooks has emerged.However, these materials almost exclusively focus on linguistic translation and scarcely consider multimodal translation.Even those that discuss audiovisual translation predominantly address technical or linguistic translation issues, rarely highlighting the role of non-linguistic modalities in translation.The necessity for textbooks that specifically cater to multimodal translation and emphasize its distinctive features is undeniable if the goal is to train proficient multimodal translators.
When compiling multimodal translation teaching materials, the primary task is to determine the specific content of the textbooks to ensure comprehensive coverage of both the theory and practice of multimodal translation.Similar to traditional language translation textbooks, multimodal translation materials should also be divided into theoretical and practical sections to provide students with a structured and comprehensive learning experience.
Regarding translation theory, as Tan Zaixi (2000:25) suggested, "a complete set of translation theory should include five components: (1) clarifying the essence of translation; (2) describing the translation process; (3) defining translation principles and standards; (4) describing translation methods; (5) explaining various contradictions in translation."Understanding these basic theoretical aspects is essential for students to gain a profound understanding of the translation activity.Therefore, the theoretical part of multimodal translation materials could be organized around these components: the essence of multimodal translation; the process of multimodal translation; principles and standards of multimodal translation; methods of multimodal translation; contradictions in multimodal translation.Since traditional translation textbooks have already introduced the essence, process, principles, standards, methods, and contradictions of general translation, students are already somewhat familiar with these aspects.The multimodal translation material should, therefore, emphasize how multimodal translation differs from traditional linguistic translation.For example, the essence section should highlight the nature of non-linguistic signs in conveying meaning, the process section should focus on the coordination and transformation of text and images, principles and standards should highlight the use of nonlinguistic signs and the standards for conveying meaning, methods should emphasize how to combine text and images more effectively, and contradictions should reveal the conflicts between text and images during the translation process.
The practical section of multimodal translation materials should primarily focus on the application of multimodal translation strategies and methods, addressing specific issues encountered in multimodal translation practice.It should include a collection of case studies from various fields, highlighting features distinct from monomodal language translation.For instance, the challenges of translating scientific literature, where the original images may pose new constraints or require innovative translation solutions (Ketola, 2018), should be discussed.Multimodal translation involves handling both linguistic and non-linguistic signs.It emphasizes not merely the linguistic conversion between the source and target texts but centers on the modal conversion between them (Wu Yun, Mou Yiwu, 2022).Thus, the practical section should guide students through innovative methods and strategies that integrate non-linguistic signs, fostering flexibility in multimodal translation practice.
In designing the practical part of multimodal translation teaching materials, it is crucial to recognize that this is not merely a technical process of language conversion but a comprehensive translation practice involving multiple communication modes and sign systems.This section should focus on the practical application of multimodal translation strategies and methods, addressing specific problems encountered in real translation tasks, and should include extensive case analyses demonstrating how these strategies and methods are applied across different domains.For example, the translation of the comic "Zhuangzi Speaks" could serve as a typical case study in multimodal translation materials.

Multimodal Translation Practices Illustrated: The Case of Zhuangzi Speaks
In the translation of Zhuangzi Speaks Brian primarily enhanced the detail in the imagery presentation and the connections between successive images.For instance, the introductory section about Zhuangzi provides an example: Original text: Zhuangzi, whose given name was Zhou, was a person from the Song state during the Warring States period.It was an era dominated by the strong bullying the weak, widespread violence and pain, and the harsh reality of the world was an endless abyss.The wise men buried beneath the mounds of yellow earth-were they great, or were they insignificant?(Tsai Chih Chung, 2013d: 3) This passage introduces Zhuangzi's identity and the chaotic backdrop of his time, highlighting the turbulence of the era.The corresponding image close-up depicts a chaotic battlefield with soldiers in combat and civilians suffering greatly, with Zhuangzi standing to the side, his face showing sorrow.Between the first and second images, Zhuangzi's figure is emphasized, his tears clearly conveying his sadness and helplessness, which sets the foundation for the following narrative.The comic's second segment states: "From then on, Zhuangzi's gaze shifted away from the mortal world, and he contemplated the infinite realms of time and space." To make the narrative flow more cohesive between the two images and to accentuate Zhuangzi's response to the turmoil, Brian made additions in the translation of these images:  Enhanced Details: Brian augmented the graphical details to emphasize the emotional depth of Zhuangzi's character.The detailed depiction of the chaotic scenes and Zhuangzi's expressive sadness deepens the reader's understanding of his philosophical detachment and existential musings.
 Image Connection: By enhancing the continuity between images, Brian ensured that the transition from Zhuangzi's experience of worldly sorrow to his philosophical detachment was visually and narratively smooth.This method of visual storytelling is crucial in multimodal translation, as it employs both textual and visual modalities to maintain the integrity of the narrative and emotional resonance.
These modifications not only enrich the translation but also exemplify how multimodal translation practices can be utilized to adapt and convey complex philosophical ideas effectively in different cultural contexts.This case demonstrates the potential of multimodal translation to enhance the reader's experience by integrating and amplifying the interplay between text and images, thus faithfully reflecting the original's thematic and emotional layers.
The practical part of multimodal translation teaching materials should encourage students to analyze such cases to understand the interdependencies of textual and visual elements and to develop strategies for their own translation work.This involves not just a linguistic transfer but a holistic approach to translation that considers all communicative modes to ensure the translated version resonates well with the target audience, preserving both the content's essence and its aesthetic appeal.
The name of our hero is Zhuang Zhou, and like all Chinese name, the surname comes first, followed by the given name.
To show respect for his vast wisdom, we add the word zi to his surname, just like Kongzi (Confucius), Menzi (Mencius) and Laozi.Zhuangzi lived during the fourth century B. C, a time known as the Warring States period in China.This was a period of disunity in which rival nations battled constantly for more land and greater power.As a result, it was also a time of widespread death and destruction.Zhuangzi saw this and was deeply saddened by it.(Cai Zhizhong, 2013: 3)

Conclusion
Today, we inhabit a highly visual world where images are ubiquitous, and ideas and information increasingly rely on visual forms for dissemination (Kress, 2003).Multimodal discourse has become the mainstream in information transmission.Consequently, in the field of translation, multimodal translation is poised to play an increasingly significant role.To meet the translation demands of this era, it is essential to foster the development of multimodal translation talent.The training of such talent necessitates the support of multimodal translation teaching materials.
The compilation of multimodal translation teaching materials needs to reflect the distinct characteristics of multimodal translation as compared to single-language modality translation.This involves integrating theory and practice to cultivate students' awareness of multimodal translation and enhance their multimodal translation skills.By presenting a plethora of cases, these materials should showcase various multimodal translation strategies and methods, thus facilitating the training of multimodal translation professionals.
In translating "Zhuangzi Speaks," Brian's adaptations provide a clear illustration of how translators can bridge cultural and contextual gaps in multimodal texts.The original brief mention of Zhuangzi's name and background by Tsai Chih Chung was expanded by Brian to clarify and emphasize Zhuangzi's stature as a sage for a Western audience unfamiliar with ancient Chinese scholarly naming conventions.Similarly, Brian's enhancements to the descriptions of the chaotic Warring States period and the emotional impact of such turmoil on Zhuangzi enriched the narrative coherence and emotional engagement for readers less familiar with the concept of homeland as understood in Chinese culture.By adding transitional phrases like "As a way out," Brian closely linked Zhuangzi's profound sadness to his subsequent philosophical detachment, enhancing the textual continuity and deepening the philosophical insights.
These adaptations exemplify effective multimodal translation practices, where the translator not only transposes text but also recalibrates and enriches visual and textual elements to resonate within the cultural and cognitive frameworks of the target audience.This case underscores the necessity for teaching materials that not only instruct on how to handle linguistic elements but also guide students in managing the interplay between text and imagery, sound, and other modalities.This holistic approach is critical in preparing translators to tackle the complex challenges of modern multimodal communication, ensuring they can deliver translations that are not only accurate but also culturally and contextually attuned.

Municipal
Education Commission: Multimodal Translation and International Communication of Chinese Cultural Classics in the Era of Digital Reading (Project Number: 24SKGH061) and the Chongqing Municipal Education Planning Project: Study on the Training of Multimodal Translation Talent in the Digital Age (Project Number: 2020-GX-278).