A Methodological Framework for Collecting and Processing Yi Monolingual and Yi-Chinese Bilingual Corpora
DOI:
https://doi.org/10.54097/cj7vh135Keywords:
Yi Character, Yi-Chinese Bilingualism, Corpus Collection, Corpus ProcessingAbstract
Addressing the challenges of collecting and processing Yi monolingual and Yi-Chinese bilingual corpora, this study proposes leveraging Optical Character Recognition and web crawler technologies to collect corpus from printed publications and the World Wide Web. The process encompasses corpus denoising, proofreading, bilingual sentences alignment, deduplication, and word segmentation, culminating in corpus storage. The theoretical framework presented herein offers valuable insights for the collection and processing of corpus, while the constructed corpora can serve as a resource for research in Yi character information processing, Yi-Chinese translation, and Yi-Chinese bilingual language ontology.
Downloads
References
[1] Haipeng S, Xueyan D, Jian S, et al. A Method for Detecting and Recogni-zing Yi Character Based on Deep Learning [J]. Computers, Materials & Con-tinua,2024,78(2):2721-2739. DOI:10.32604/CMC.2024.046449.
[2] Shang W, Chen L, Xiang W. Yi printed character recognition based on de-ep learning[J].Procedia Computer Science, 2024, 242584-591.DOI:10.1016/J.PROCS.2024.08.111.
[3] Cheng Ping W. The Research on Method and Difficulty Analysis of Autom-atic Word Segmentation in Yi Language for Information Processing[J]. The Frontiers of Society, Science and Technology,2019,1(8).
[4] Chengping W. The Research and Construction of Yi Corpus for Informatio-n Processing[J]. International Journal of New Developments in Engineering and Society,2019,3(4).
[5] https://github.com/tesseract-ocr/.
[6] Mitchell, R. Web Scraping with Python, 2nd ed.; No Starch Press: Sebasto-pol, CA, USA, 2018; pp. 33–48.
[7] Qu Y H, Zhao W. A Revised BMM and RMM Algorithm of Chinese Aut-omatic Words Segmentation[J].Advanced Materials Research,2011,1286(267-267):199-204.DOI:10. 4028/ www. scientific.net/AMR.267.199.
[8] https://github.com/fxsjy/jieba.
[9] Zhou Q, Zhang J. Research on the Design and Implementation of Case T-eaching in MySQL Database Course[J].Education Reform and Development,2025,7(5):145-150.DOI:10. 26689/ ERD. V7I5.10588.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Journal of Education and Educational Research
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
