Cross-Lingual Transfer Learning: Applications in Low-Resource Languages

Junfu Zhu

doi:10.54097/5wgzb327

Authors

Junfu Zhu

DOI:

https://doi.org/10.54097/5wgzb327

Keywords:

Cross-lingual Natural Language Processing, Low-resource Languages, Machine Translation, Text Classification.

Abstract

With the acceleration of globalization and the rapid advancement of information technology, cross-lingual natural language processing has become a prominent research focus. However, most languages worldwide remain low-resource and lack sufficient annotated data to train high-quality models. Cross-lingual transfer learning is an effective approach to alleviate data scarcity. It transfers knowledge learned in high-resource languages to low-resource languages and significantly improves performance on downstream tasks with little or no target-language supervision. This paper systematically reviews applications of cross-lingual transfer learning for low-resource languages with a focus on machine translation, text classification, and named entity recognition. It synthesizes technical approaches, identifies persistent challenges, and outlines future directions. Through a comprehensive analysis of the literature, this paper summarizes key techniques and application outcomes in cross-lingual transfer learning, providing researchers with practical insights and recommendations to guide future research and deployment. The goal is to promote the broader application and development of cross-lingual NLP technologies for low-resource languages.

Downloads

Download data is not yet available.

References

[1] Ruder S, Vulić I, Søgaard A. A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, 2019, 65: 569-631.

[2] Dabre R, Chu C, Kunchukuttan A. A survey of multilingual neural machine translation. ACM Computing Surveys, 2020, 53 (5): 1-38.

[3] Lado R. Linguistics across cultures; applied linguistics for language teachers. 1957.

[4] Corder S P. The significance of learner's errors. 1967.

[5] Gathercole S E, Adams A M, Hitch G J. Do young children rehearse? An individual-differences analysis. Memory & Cognition, 1994, 22 (2): 201-207.

[6] Pires T, Schlinger E, Garrette D. How multilingual is multilingual BERT?. arXiv preprint arXiv:1906.01502, 2019.

[7] Conneau A, Khandelwal K, Goyal N, et al. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116, 2019.

[8] Sennrich R, Haddow B, Birch A. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709, 2015.

[9] Edunov S, Ott M, Auli M, Grangier D. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381, 2018.

[10] Lample G, Ott M, Conneau A, Denoyer L, Ranzato M A. Phrase-based & neural unsupervised machine translation. arXiv preprint arXiv:1804.07755, 2018.

[11] Artetxe M, Labaka G, Agirre E, Cho K. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041, 2017.

[12] Johnson M, Schuster M, Le Q V, et al. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 2017, 5: 339-351.

[13] Wu H, Wang H. Pivot language approach for phrase-based statistical machine translation. Machine Translation, 2007, 21 (3): 165-181.

[14] Chen X, Sun Y, Athiwaratkun B, Cardie C, Weinberger K. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics, 2018, 6: 557-570.

[15] Shaalan K. A survey of Arabic named entity recognition and classification. Computational Linguistics, 2014, 40 (2): 469-510.