Effects of Bilingual Videos in National Innovation Project in College English Listening Teaching Under the Influences of Multi-modal Theory


  • Jiaying Huang
  • Huan Cao




Multi-modal, Listening comprehension, Empirical study, Subtitles.


Multi-modal discourse refers to the phenomenon of using multiple senses such as auditory, visual, and tactile senses to communicate through multiple means and symbolic resources such as language, images, sounds, and actions. This paper reviews the theory of multi-modal discourse and studies the application of multi-modal discourse in English teaching at home and abroad, and conducts an empirical study of a new university English listening teaching mode in a multi-modal context by using comparative analysis method, taking Y University as an example. Secondly, through comparative analysis, one control group and four experimental groups were set up , the relevant materials of this experiment are all from the impression Yue platform, and video, audio, Chinese subtitles and English subtitles were used as variables, and through analyzing the experimental results, it was found that the reasonable use of multi-modal combinations can significantly improve students’ listening comprehension, and the use of audio-visual combinations and English subtitles is more beneficial to students’ listening The experimental results showed that the use of multi-modal combinations can significantly improve students’ listening comprehension, and the use of audio-visual combinations and English subtitles is more beneficial to students' listening comprehension.


Download data is not yet available.


Baltova, I. (1999). Multisensory language teaching in a multidimensional curriculum : The use of authentic bimodal video in core French. Canadian Modern Language Review-revue Canadienne Des Langues Vivantes, 56, 31-48.

Coffey, D. (2020). Personalized multi-modal instruction: Positively impacting lives through invitational education. Journal Of Invitational Theory And Practice, 12(26), 55-69.

Fisher, D. L. (2020). Using powerpoint for ESL teaching: The internet TESL journal. 11(9), 4.

Ganapathy, M. (2016). The effects of using multi-modal approaches in meaning making of 21st century literacy texts among ESL students in a private school in malaysia. Advances in Language and Literary Studies, 7(2), 143-155.

Hung, H.T., Chiu, Y.C. J., & Yeh, H.C. (2012). Multi-modal assessment of and for learning: A theory-driven design rubric. British Journal of Educational Technology, 44(3), 400–409.

Kalantzis,M.&Cope,B. (2005). With the Learning by Designproject group. Learning by Design. Melbourne:Victo-rian Schools Innovation Commission.

Malhotra, A. Yang, C., & Feng, X. (2021). Application of constructivism and cognitive flexibility theory to build a comprehensive, integrated, multi-modal inter-professional education and practice (CIM-IPEP) program. Journal of Interprofessional Care, 14(2), 1-6.

Nagy, N. (2020). Multi-modal literacy development in a higher education english studies classroom. Journal of Visual Literacy, 39(3), 167-184.

Nouri, H., & Shahid, A. (2005). The effect of powerpoint presentations on student learning and attitudes. Global Perspectives on Accounting Education, 22(3), 53-73.

Romero, E.D., & Arévalo, C.M. (2010). Multimodality and listening comprehension: testing and implementing classroom material. Language Value, 10(2), 100-139.

Teo, P., & Zhu, J. (2018). A Multi-modal analysis of affect and attitude education in china’s english textbooks. Multi-modal Communication, 7(1), 5-15.

Weinberg, A. (2013). Virtual Misadventures: Technical problems and student satisfaction when implementing multimedia in an advanced french listening comprehension course. The CALICO Journal, 19(2), 331-357.

Fan Ling.The reform of college English teaching from the perspective of multi-modal theory[J]. Education Exploration, 2013(07):71-72.

Gu Yueguo.The learning analysis of multi-media and multi-modal[J]. Technology Enhanced Foreign Languages, 2007(2):21-22.

Hu Zhuanglin. Personalization, collaboration, modularization and hypertextualization of university English teaching[J]. Foreign Language Teaching and Research, 2004(9):31-42.

Hu Zhuanglin. Multi-modal theory in social semiotic research[J]. Language Teaching and Linguistic Studies, 2007( 1):14-22.

Hu Yongjing and Zhang Delu. An experimental study of multi-modal functions in English major listening teaching[J]. Foreign Language World, 2013(05):20-25+44.

Li, BaoHong and Yin PiAn. An empirical study on the teaching model of English in multi-modal contexts in universities[J]. Technology Enhanced Foreign Languages, 2012(06):72-75.

Liang Hao and Lu Yan. The application of multi-modal theory in English listening and reading teaching[J]. Teaching and Management, 2016(27):110-112.

Liu Ming and Hu Jiasheng. Analysis of multi-modal design concept and requirements of university foreign language audio-visual teaching materials[J].Technology Enhanced Foreign Languages, 2011(02):3-8+20.

Li Zhanzi. A social semiotic analysis of multi-modal discourse[J]. Foreign Language Research, 2003(5):34-52.

Pan Mingwei. Oral communication competence in a multi-modal perspective: Reconstructing and exploring[D]. Shanghai International Studies University, 2011.

Sun Zhinan. The application of multi-modal discourse theory in college English teaching[J]. Educational exploration, 2011(08):45-46.

Deng Qingmin. A study on the effectiveness of multi-modal audio-visual teaching mode on the development of listening and speaking ability[J]. Journal of the PLA Foreign Language Institute, 2011, 34(06):72-76+128.

Zhang Delu. Exploring a comprehensive theoretical framework for multi-modal discourse analysis[J]. Foreign Languages in China, 2009(1):35-56.

Zhang Delu and Ding Zhaofen. Exploring a multi-modal selection framework for foreign language teaching[J]. Foreign Language World, 2013(03):39-46+56.

Zhang Delu and Wang Lu. The Synergy of Multi-modal Discourse Modality and its Embodiment in Foreign Language Teaching[J]. Foreign Language Research, 2010(02):97-102.

Zhang Zhaoli. The application of multi-modal theory in junior high school English teaching[D]. Shandong Normal University, 2013.



24 May 2023

How to Cite

Huang, J., & Cao, H. (2023). Effects of Bilingual Videos in National Innovation Project in College English Listening Teaching Under the Influences of Multi-modal Theory. International Journal of Education and Humanities, 8(3), 188–195. https://doi.org/10.54097/ijeh.v8i3.8703