Document-level Relation Extraction based on Graph Convolutional Neural Networks

Pengfei Song; Haoyue Lu

doi:10.54097/tgnshp08

Authors

Pengfei Song
Haoyue Lu

DOI:

https://doi.org/10.54097/tgnshp08

Keywords:

Document-level Relation Extraction, Graph Convolutional Network, DocRED

Abstract

The objective of extracting relations between document components lies in identifying the Within a single document, the focus often lies on understanding the connections between different entities. This type of analysis goes beyond individual sentences, as it demands an understanding of how information from multiple sentences interacts to form these connections. Over recent years, the importance of exploring relationships involving several entities simultaneously has grown significantly. To advance the field of studying such connections across entire documents, a novel collection of data points, known as DocRED, has been introduced. Currently, the standard approach for this task involves using BiLSTM networks to process the entire document as a whole. However, this method struggles to effectively capture the intricate relationships that exist among various entities. To overcome this limitation, a new model designed for document-level relationship identification has been developed, which leverages Graph Convolutional Networks (GCN). GCNs are particularly useful here because they can gather information from surrounding entities, allowing for a more detailed modeling of their interactions. The proposed approach starts by identifying coreferential links to gather features that represent the relationships between pairs of entities. These features are then analyzed using GCN to construct a graph structure that represents the entire document, ultimately revealing the complex interactions between different entities. Testing this model on the large-scale DocRED dataset from Tsinghua University has demonstrated its strong performance in this challenging task.

Downloads

Download data is not yet available.

References

[1] Chinchor N, Marsh E, MUC-7 information extraction task definition[C]// Proceedings of the 7th Message Under standing Conference(MUC-7). Stroudsburg, PA, USA: Association for Computational Linguistics, 1998: 359- 367.

[2] Bayu Distiawan Trisedya, Gerhard Weikum, Jianzhong Qi, and Rui Zhang. 2019. Neural relation extraction for knowledge base enrichment. In ACL, pages 229-240, Florence, Italy. ACL.

[3] Mo Yu, Wenpeng Yin, Kazi Saidul Hasan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2017. Improved neural relation detection for knowledge base question answering. In ACL, pages 571-581, Van-couver, Canada. ACL.

[4] Tom Young, Erik Cambria Cambria, Iti Chaturvedi, Minlie Huang, Hao Zhou, and Subham Biswas. 2018. Augmenting end-to-end dialog systems with commonsense knowledge. In AAAI.

[5] Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. 2019. DocRED: A large-scale document-level relation extraction dataset. In Pro- ceedings ofACL 2019.

[6] Gan L X, Wan C X,Liu D X, et al. Chinese Named Entity Relation Extraction Based on Syntatic and Semantic Features [J]. Journal of Computer Research and Development, 2016, 53(2):284-302.

[7] Choi S P ,Lee S ,Jung H et al.An intensive case study on kernel-based relation extraction[J]. Multimedia Tools & Appli-cations ,2014 ,71(2): 741-767.

[8] Zeng D ,Liu K ,Lai S et al.Relation Classification via Convolutional Deep Neural Network[C]//Proceedings of the 25th Inter national Conference on Computational Linguistics. 2014:2335-2344.

[9] Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1746-1751. Association for Computational Linguistics.

[10] Sunil Kumar Sahu and Ashish Anand. 2018. Drug- drug interaction extraction from biomedical texts using long short-term memory network. Journal of Biomedical Informatics, 86:15 -24.

[11] Li F L,Ke J .Research Progress of Entity Relation Ex traction Base on Deep Learning Framework [J].Information Science, 2018, v36(3):169-176.

[12] Mintz M ,Steven B,Rion S,et al .Distant super vision for relation extraction without labeled data[C]//Proceedings of Joint Conference of the Meeting of the ACL .Stroudsburg:As- sociation for Computational Linguistics,2009:1003-1011.

[13] Hoffmann R,Zhang C,Ling X,et al. Knowledge-based weak supervision for information extraction of overlapping relations [C]//Proceedings of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2011:541-550.

[14] Surdeanu M,T Ibshirani J,Nallapat I R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of Joint Conference on Empirical Methods in Natural Language Processing and Computational Natura Language Learning. 2012: 455-465.

[15] Benjamin R ,Dietrich K .Combining Generative and Discriminative Model Scores for Distant Supervision[C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.2013:24-29.

[16] Zeng D ,Liu K ,Chen Y ,et al .Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of Conference on Empirical Methods in Natural Language Processing.2015:1753-1762.

[17] Lei K ,Chen D ,Li Y ,et al.Cooperative Denoising for Distantly Supervised Relation Ex traction[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018: 426-436.

[18] Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen tau Yih. 2017. Cross-sentence n-ary relation extraction with graph lstms. Transactions of the Association for Computational Linguis- tics, 5:101-115.

[19] H.Wang, C. Focke, R. Sylvester, N. Mishra, andW. Wang, Fine-tune bert for DocRED with two-step process,2019, arXiv:1909.11898. [Online]. Available: http:// arxiv. org/ abs/ 1909. 11898.

[20] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need,in Proc. Adv. Neural Inf. Process. Syst., 2017, pp. 5998–6008.

[21] Thomas N Kipf and Max Welling. 2017. Semi- supervised classification with graph convolutional networks. In International Conference on Learning Representations.

[22] Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for se- mantic role labeling. In Proceedings of Conference on Empirical Methods in Natural Language Processing, pages 1506–1515. Association for Computational Linguistics.[. 2017. Graph-based neural multi-document summarization. In Proceedings of the 21st Conference on Computational Linguistics.

[23] Michihiro Yasunaga, Rui Zhang, Kshitijh Meelu, Ayush Pareek, Krishnan Srinivasan, and Dragomir Radev. 2017. Graph-based neural multi-document summarization. In Proceedings of the 21st Conference on Computational Natural Language Learning, pages 452–462. Association for Computational Linguistics.

[24] Shikhar Vashishth, Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha Talukdar. 2018. Dating documents using graph convolution networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1605–1615. Association for Computational Linguistics.

[25] Yuhao Zhang, Peng Qi, and Christopher D Manning. 2018. Graph convolution over pruned dependency trees improves relation extraction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2205–2215. Association for Computational Linguistics.

[26] Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532-1543.

[27] Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017).

[28] Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING, pages 2335–2344.

[29] Sepp Hochreiter and Jurgen Schmidhuber.1997.Long short-term memory. Neural Computation, 9:1735–1780.

[30] Rui Cai, Xiaodong Zhang, and Houfeng Wang. 2016. Bidirectional recurrent convolutional neural network for relation classification. In Proceedings of ACL, pages 756-765.

[31] Daniil Sorokin and Iryna Gurevych. 2017. Contextaware representations for knowledge base relation extraction. In Proceedings of EMNLP, pages 1784-1789.

[32] Hong Wang, Christfried Focke, Rob Sylvester, Nilesh Mishra, and William Wang. 2019. Fine-tune bert for docred with two-step process. arXiv preprint arXiv:1909.11898.