An Overview of Research on Multimodal Teaching and Learning

Yan Lv; Wenqian Ni; Yujiao Tan

doi:10.54097/qp4jg551

Authors

Yan Lv
Wenqian Ni
Yujiao Tan

DOI:

https://doi.org/10.54097/qp4jg551

Keywords:

Multimodal Teaching and Learning, Multimodal Discourse Analysis, Multimodal Classroom Teaching, Multimodal Textbook Analysis

Abstract

With the development of modern information technology, there emerges an increasing trend of multimodal teaching and learning. Teachers are required to employ a wide range of materials such as textbooks, diagrams, photographs and drawings to facilitate students’ learning. Thus, the significance of applying multimodal textbook discourse into classroom setting is also becoming increasingly pronounced. This article takes a retrospective look at the related previous studies on multimodal teaching and learning, which cover a wide variety of perspectives, including the multimodal discourse analysis, the multimodal classroom teaching and the multimodal textbook analysis. Based on the literature review, the deficiencies of current multimodal studies and suggestions for further research have been proposed in the last part of the article. The current study can give important enlightenment to the research and practice of multimodal teaching and learning, especially with the aid of the multimodal textbook discourse analysis.

Downloads

Download data is not yet available.

References

Abbsa, M. B. The efficacy of multimodal vs. print-based texts for teaching reading comprehension skills to lranian school third graders[J]. International Journal of Language Learning and Applied Linguistics World, 2014(3):365-380.

Barthes, R. Rhetoric of the image [A]. in R. Barthes (ed.) Image Music Text[C]. London: Fontana,1977: 32–51.

Bateman, J. Using multimodal copora for empirical research [A]. in C. Jewitt (ed). Routledge Handbook of Multimodal Analysis[C]. London: Routledge, 2014: 238-252.

Bezemer, J. & Kress G. Writing in multimodal texts: A social semiotic account of designs for learning [J]. Written Communication, 2008, 25 (2):166–195.

Bezemer, J., & Kress, G. The textbook in a changing multimodal landscape[A]. in N.M. Klug & H. Stöckl (eds). Language in Multimodal Contexts [C]. New York: De Gruyter, 2015: 21–28.

Chen Z.F.& Yang Y.& Cao Z.H. The construction of the dynamic balance teaching mode in college English class on the basis of multimodality and multimedia [J]. Journal of Xidian University (Social Science Edition), 2014(04):110-115.

Evan, M. A. & Watson C. & Willows D.M. A naturalistic inguiry into illustrations in instructiuonal textbooks [J]. The Psychology of lllustration. Instructional Issues. New York: Springer US, 1978(02): 86-115.

Germán C. A Multimodal and ethnographic approach to textbook discourse [A]. in A. Kay & L. O’Halloran (eds.), Routledge Studies in Multimodality [C]. New York: Routledge, 2022: 1-142.

Halliday, M.A.K. & Hasan, R. Language, context and text: Aspects of language in a social-semiotic perspective [M]. Victoria: Deakin University Press,1985.

Hu Z.L. Multimodalization in social semiotics [J]. Foreign Language Teaching and Research, 2007(01):1-10.

Jewitt, C. Multimodal analysis [A]. in Georgakopoulou A. & T. Spilioti (eds.), The Routledge Handbook of Language and Digital Communication [C]. London and New York: Routledge, 2016:69–84.

Kress, G., & Van Leeuwen, T. Reading image: The grammar of visual design. [M]. New York: Rouledge,1996.

Kress, G. Multimodality: A sociosemiotic approach to contemporary communication [M]. London: Routledge, 2010.

Li Z.Z. Social semiotic approach to multimodal discourse [J]. Foreign Language Research, 2003,81(05):1-8.

Machin, D. Introduction to multimodal analysis [M]. London and New York: Bloomsbury, 2007.

Martine R. & Salway A. A system for image-text relations in new (and old) media [J]. Visual Communication, 2005(03): 337-371.

McMurtrie,R.J. Observing, recording, visualising and interpreting visitors’ movement patterns in art museums: A mixed method approach [J]. Multimodality & Society, 2022, 2(2): 93-113.

Meneses, A. & Escobar, J. & Veliz, S. The effects of multimodal texts on science reading comprehension in Chilean fifth-graders: text scaffolding and comprehension[J]. International Journal of Science Education, 2018(18):2226-2244.

New London Group. A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review[J]. 1996, 66(1): 60-93.

O'Halloran, K. L., & Smith, B. A. Multimodal studies [A]. in K. L. O'Halloran & B. A. Smith (eds.), Multimodal Studies: Exploring Issues and Domains[C]. London: Routledge, 2011: 1-13.

Ravelli, L. J. & McMurtrie R. J. Multimodality in the built environment: Spatial discourse analysis [M]. New York: Routledge, 2016.

Wang, R. H. & Jiang, G.Y. Multimodal foreign language teaching: Blending of image and language resources[J]. Journal of Schooling Studies, 2015,12(03):84-90.

Royce, T. Multimodality in the TESOL classroom: Exploring visual-verbal synergy [J]. TESOL Quarterly, 2002, 36(2): 191-205.

Scollon, R.,& Levine, P. Multimodal discourse analysis as the confluence of discourse and technology[A]. in P. Levine & R. Scollon (eds.), Discourse & Technology: Multimodal Discourse Analysis [C]. Washington: Georgetown University Press, 2004: 1–6.

Segers, E.& Verhoeven,L.& Hulstiin-Hendrikse, N. Cognitive processes in children's multimedia text learning [J]. Applied Cognitive Psychology, 2008,22(3):375-387.

Sinda M.&Ahmad J. A Multimodal discourse analysis of “Lathi” song [J]. International Journal of Linguistics, Literature and Translation, 2022, 5(1): 66-72.

Song Q.W. The empirical research on multimodality and the cultivation of multi-dimensional reading and writing abilities in college English [J]. Foreign Languages Research, 2013,138 (2): 55–59.

Stein, P. Rethinking resources multimodal pedagogies in the ESL Classroom [J]. TESOL Quarterly, 2000,34 (2): 333-336.

Tuomo H. The Structure of multimodal documents: An empirical approach [M]. New York: Routledge, 2015.

Torres, G. P. 'Reading’ world link: A visual social semiotic analysis of an EFL textbook [J]. International Journal of English Language Education, 2015, 3(1):239-253.

Xie N.N. Analysis of logico-semantic expansion of image-text relations in Grimm’s Tales [J]. Foreign Language Teaching, 2014, 35(01): 21-25.

Zhang D.L. & Ding Z.F. Exploration of the multimodal selective framework[J]. Foreign Language World, 2013, 156 (03): 39-56.

Zeng F.B. A study on the construction of image-text relations and its models in mulitmodal discourse: Comments and ananlysis on the three image-text relations theories[J]. Foreign Language and Literature, 2010, 26(04):60-64.

Zhang D.L. On the models of modal integration in multimodal discourse construction[J]. Modern Foreign Languages,2023, 46(04):439-451.

Zhu Y.S. Theory and methodology of multimodal discourse analysis[J]. Foreign Language Research, 2007,138(05):82-86.