Journal of Shanghai University >
Distant supervision for relation extraction via attention CNNs
Received date: 2019-08-20
Online published: 2019-10-18
The process of relation extraction is a significant step in several information extraction systems designed to mine structured facts from text. However, two problems surface when traditional distant supervision methods are employed to conduct the entity relation extraction task. First, the distant supervision heuristic aligns the text in the corpus using existing knowledge marked with entities and relations, after which the alignment results are treated as annotation data; this leads to inevitable labeling errors. Second, current statistical methods rely extensively on natural language processing tools to extract features, and the noise accumulating during the entire process significantly affects the extraction results. In this study, an end-to-end, attention mechanism-based convolutional neural network (CNN) is proposed. First, the attention mechanism is added to the input layer for automatic detection of more subtle clues and learning of parts of sentences that are relevant to relation extraction. Second, the sentence is encoded based on the position feature and word feature, a piecewise CNN (PCNN) is used to extract sentence features and classify relationships, and finally a max-margin loss function with a higher efficiency is used on the network. The accuracy of this method when used on the New York Times dataset is 2.0% higher than that of the classical PCNN+MIL model, and 1.0% higher than that of the classical APCNN+D model. The experimental results therefore demonstrate excellent accuracy of the proposed model when compared with that of other baselinemodels.
XING Yixue, ZHU Yonghua, GAO Haiyan, ZHOU Jin, ZHANG Ke . Distant supervision for relation extraction via attention CNNs[J]. Journal of Shanghai University, 2021 , 27(5) : 983 -992 . DOI: 10.12066/j.issn.1007-2861.2197
| [1] | Liu Q, Li Y, Duan H, et al. A survey of knowledge mapping construction techniques[J]. Journal of Computer Research and Development, 2016, 53(3): 582-600. |
| [2] | Mintz M, Bills S, Snow R. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011. |
| [3] | Zeng D, Liu K, Chen Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015: 1753-1762. |
| [4] | Santos C N D, Xiang B, Zhou B. Classifying relations by ranking with convolutional neuralnetworks[J]. Computer Science, 2015, 86: 132-137. |
| [5] | Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]// The 25th International Conference on Computational Linguistics: Technical Papers. 2014: 2335-2344. |
| [6] | Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees[C]// Proceedings of the Meeting of the Association for Computational Linguistics. 2017: 917-928. |
| [7] | Liu T, Wang K, Chang B, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]// Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2017: 1790-1795. |
| [8] | Lin Y, Shen S, Liu Z, et al. Neural relation extraction with selective attention overinstances[C]// Proceedings of the Meeting of the Association for Computational Linguistics. 2016: 2124-2133. |
| [9] | Ji G L, Liu K, He S Z, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]// Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. 2017: 3060-3066. |
| [10] | Li M Y, Yang J. Open Chinese entity relationship extraction method based on dependency parsing[J]. Computer Engineering, 2016, 42(6): 201-207. |
| [11] | Riedel S, Yao L, Mccallum A. Modeling relations and their mentions without labeledtext[C]// Machine Learning and Knowledge Discovery in Databases. 2010: 148-163. |
| [12] | Hoffmann R, Zhang C, Ling X. Knowledge-based weak supervision for information extraction of overlapping relations[C]// Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011: 541-550. |
| [13] | Surdeanu M, Tibshirani J, Nallapati R. Multi-instance multi-label learning for relation extraction[C]// Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012: 455-465. |
| [14] | Finkel J R, Grenager T, Manning C. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling[C]// Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 2005: 363-370. |
| [15] | Mintz M, Bills S, Snow R. Distant supervision for relation extraction without labeleddata[C]// Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011. |
| [16] | Jiang X, Wang Q, Li P, et al. Relation extraction with multi-instance multi-label convolutional neural networks[C]// The 26th International Conference on Computational Linguistics: Technical Papers. 2016: 1471-1480. |
| [17] | Yu X K, Chen L, Guo J, et al. Relationship extraction method combining clause-level remote supervision and semi-supervised integration learning[J]. Pattern Recognition and Artificial Intelligence, 2017, 30(1): 54-63. |
| [18] | Jiao L C, Yang S Y, Liu F, et al. Neural network seventy years: retrospect and prospect[J]. Chinese Journal of Computers, 2016, 39(8): 1697-1716. |
| [19] | Ren X, Wu Z, He W, et al. CoType: joint extraction of typed entities and relations with knowledge bases[C]// Proceedings of the 26th International Conference on World Wide Web. 2017: 1015-1024. |
| [20] | Yang J F, Yu Q B, Guan Y, et al. A survey of research on electronic medical record named entity recognition and entity relationship extraction[J]. Acta Automatica Sinica, 2014, 40(8): 1537-1562. |
| [21] | Wang L, Cao Z, Melo G D, et al. Relation classification via multi-level attention CNNs[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1298-1307. |
/
| 〈 |
|
〉 |