Chinese Named Entity Recognition based on ERNIE
DOI:
https://doi.org/10.54097/fcis.v3i2.6912Keywords:
Named Entity Recognition, Enhanced Representation through Knowledge Integration, Gated Recurrent Unit, Conditional Random FieldAbstract
The traditional named entity recognition model based on neural network uses static word vector, which can’t represent the ambiguity of the word in the context. The ERNIE-BiLSTM-CRF model is proposed. The ERNIE pre-training model can output different word vectors for different contexts by using multiple layers of Transformer, obtaining dynamic word vectors that contain overall sequence information. Secondly, the word vectors are input into the BiLSTM layer, which can obtain sentence context information through forward and backward LSTM and obtain more sentence features, thereby improving the model's effectiveness. Finally, the sequence is labeled through the CRF layer to obtain the globally optimal labeling information and complete the named entity recognition task. The experimental results show that compared with the traditional model, the F1 score of this model has significantly improved.
Downloads
References
MARRERO M, URBANO J, S Á NCHEZ-UADRADO S, et al. Named Entity Recognition: Fallacies, Challenges, and Opportunities [J]. Computer Standards and Interfaces, 2013, 35 (5): 482.
Zhang Libang Chinese electronic medical record segmentation and name entity mining based on semi supervised learning [D] The following is: [Master's Thesis] Harbin: Harbin Institute of Technology, 2014
Chinchor N. MUC-6 Named Entity Task Definition (Version 2.1) [C]. Proceedings of the 6th Conference on Message Understanding, Columbia, Maryland, 1995: 142-194.
LDC Corporation. Entity Detection and Tracking Phase 1 Experimental Study Task Definition [EB/OL].
https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/edt-phase1-v2.2.pdf , 2017-03-10.
Sang, Eric, Meldfen. CoNLL-2003 Shared Task Introduction: Language Independent Named Entity Recognition [C]. Calculate natural language learning progress. Association for Computational Linguistics, 2003:142-147.
Doddington G. Tasks, Data, and Evaluation of Automated Content Extraction (ACE) Programs [J]. Proc Lrec, 2004, 34(3): 123-153.
Liu Liu, Wang Dongbo Overview of Named Entity Recognition Research [J]. Journal of Information Technology, 2018, 37 (03): 329-340
Silviu Cucerzan, David Yarowsky. Language independent named entity recognition combining morphological and contextual evidence [C]. Proceedings of the 1999 SIGDAT EMNLP and VLC Joint Meeting, 1999:90-99.
Li D, Savova G, Kipper-schuler K. Conditional Random Fields and Support Vector Machines for Recognition of Named Obstacle Entities in Clinical Text [C]. Proceedings of the 2008 Symposium on Current Trends in Biomedical Natural Language Processing. 2008:94-95.
Jiang M, Chen Y, Liu M. Research on machine learning based methods to extract clinical entities and their assertions from a large number of abstracts [J]. Journal of the American Medical Informatics Association, 2011, 18 (5): 601-606.
Li L, Mao T, Huang D, et al. A Hybrid Model for Chinese Named Entity Recognition [C]. Proceedings of the 5th SIGHAN Symposium on Chinese Language Processing. Strauss burg: Computational Linguistics Association, 2006:72-78.
Cucerzan S. Large scale named entity disambiguation based on Wikipedia data [C]. Proceedings on Empirical Methods in Natural Language Processing. Czech Republic: Prague, 2007:708-716.
Ronan Collabert, Jason Weston, L é on Bottou, et al. Natural language processing starts (almost) from scratch [J]. Journal of Machine Learning Research, 2011, 12 (76): 2493-2537.
Sahu S K, Anand A. Recursive Neural Network Model for Disease Name Recognition Using Domain Invariant Features [C]. Proceedings of the 54th Annual Conference of the Computational Linguistics Association. Strauss burg: Computational Linguistics Association, 2016: 2216-2225.
Vaswani A, Shazeer N, Parmar N, etc. Attention is what you need [J]. ar Xiv, 2017, (1706): 37-62.
Yan H, Deng B, Li X, et al. TENER: Adaptive converter encoder for name entity recognition [J]. ar Xiv, 2019, (1911): 44-74.
Bharadwaj A, Mortensen D, Dyer C, etc. Speech Perceptual Neural Model for Named Entity Recognition in Low Resource Transfer Environments [C]. Proceedings of the 2016 Conference on Empirical Methods of Natural Language Processing. Strauss burg: Computational Linguistics Association, 2016:1462-1472.
Devlin J, Chang M W, Lee K, et al. Bert: Pre training of deep bidirectional converters for language understanding [J]. ar Xiv, 2018, (1810): 48-50.
Liu Y H, Ott M, Goyal N, et al. Roberta: A robust optimization Bert pre training method [EB/OL].
https://arxiv.org/abs/1907.11692 , 2019-07-26.
Lan Z, Chen M, Goodman S, et al. ALBERT: Simplified BERT [EB/OL] for language representation self supervised learning.
https://openreview.net/pdf?id=H1eA7AEtvS , 2019-07-26.
Wang Naiyu, Ye Yuxin, Liu Lu, Feng Lizhou, Bao Tie, Peng Tao. Research on Language Models Based on Deep Learning Progress [J]. Journal of Software, 2021, 32 (04): 1082-1115
DongL, YangN, WangW, et al. A unified language model for pre training and generation of natural languages [EB/OL].
https://arxiv.org/abs/1905.03197 ,2019-0 5-08.
Song K, Tan X, Qin T, et al. MASS: Language generated masking sequence to sequence pre training [EB/OL].
https://arxiv.org/pdf/1905.02450.pdf , 2019-06-21.
Tsai H, Riesa J, Johnson M, et al. A small practical BERT model for sequence tagging [C]. Procedure. 2019 Conference on Natural Language Processing Experiences and Methods and the Ninth International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019: 3623 – 3627.
Lu W, Dan R. Joint reference extraction and classification using reference hypergraphs [C]. Proceedings of the 2015 Conference on Empirical Methods of Natural Language Processing, 2015:857-867.
Muis A O, Lu W. Mark gaps between words: Use reference separators to identify overlapping references [C]. Proceedings of the 2017 Conference on Empirical Methods of Natural Language Processing, 2017:2608-2618.
Ju M, Miwa M, Ananiadou S. Neural hierarchical model for nested named entity recognition [C]. Proceedings of the 2018 North American Branch of the Computational Linguistics Association: Human Language Technology, Volume 1 (Long Paper), 2018: 1446-1459.
Jue W, Shou L, Chen K, et al. Pyramid: A Layered Model for Nested Named Entity Recognition[C]. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020:5918-5928.
Li Zhoujun, Fan Yu, Wu Xianjie. Overview of Research on Pretraining Technology for Natural Language Processing [J]. Computer Science, 2020,47 (03): 162-173
Sun Y, Wang S, Li Y, et al. ERNIE 2.0: A continual pre-train⁃ing framework for language understanding[J]. Proceedings of theAAAI Conference on Artifical Intelligence, 2020, 34(5): 8968-8975.
Luo Xiao. Overview of Natural Language Processing Research Based on Deep Learning [J]. Intelligent Computers and Applications, 2020, 10 (4): 133-137. Luo Xiao A survey of natural language processing based on deep learning[J]. Intelligent Computer and Application, 2020, 10(4): 133-137.
Downloads
Published
Issue
Section
How to Cite
