研究论文

基于注意力机制的远程监督实体关系抽取

展开
  • 1.上海大学 上海电影学院, 上海 200072
    2.上海大学 生命科学学院, 上海 200444
朱永华(1967—), 男, 副教授, 博士, 研究方向为计算机、人工智能、数据科学. E-mail: zyh@shu.edu.cn

收稿日期: 2019-08-20

  网络出版日期: 2019-10-18

基金资助

"十三五"国家重点研发计划项目(2017YFD0400101)

Distant supervision for relation extraction via attention CNNs

Expand
  • 1. Shanghai Film Academy, Shanghai University, Shanghai 200072, China
    2. School of Life Sciences, Shanghai University, Shanghai 200444, China

Received date: 2019-08-20

  Online published: 2019-10-18

摘要

关系抽取是许多信息抽取系统中的一个关键步骤, 旨在从文本中挖掘结构化事实. 在应用传统的远程监督方法完成实体关系抽取任务时存在 2 个问题: ① 远程监督方法将语料库中的文本与已标注实体和实体间关系的知识库启发式地对齐, 并将对齐结果作为文本的标注数据, 这必然会导致错误标签问题; ② 目前基于统计学的方法过于依赖自然语言处理工具, 提取特征处理过程中生成的噪声积累严重影响抽取结果. 为了解决远程监督存在的弊端, 提出了一种基于注意力机制的端到端的分段循环卷积神经网络(convolutional neural network, CNN)模型. 为了检测更加细微的特征, 在网络输入层添加了注意力机制, 自动学习句子中与关系抽取相关的内容; 基于位置特征和词向量特征对句子进行编码, 并使用分段卷积神经网络(piecewise CNN, PCNN)抽取句子特征进行分类, 在网络中使用了效率较高的最大边界损失函数来衡量模型的性能. 该方法在 New York Times (NYT)数据集上的准确率比经典的 PCNN+MIL 模型提高了 2.0%, 比经典的 APCNN+D 模型提高了 1.0%, 与其他几种基线模型相比, 该模型准确率表现出色.

本文引用格式

邢毅雪, 朱永华, 高海燕, 周金, 张克 . 基于注意力机制的远程监督实体关系抽取[J]. 上海大学学报(自然科学版), 2021 , 27(5) : 983 -992 . DOI: 10.12066/j.issn.1007-2861.2197

Abstract

The process of relation extraction is a significant step in several information extraction systems designed to mine structured facts from text. However, two problems surface when traditional distant supervision methods are employed to conduct the entity relation extraction task. First, the distant supervision heuristic aligns the text in the corpus using existing knowledge marked with entities and relations, after which the alignment results are treated as annotation data; this leads to inevitable labeling errors. Second, current statistical methods rely extensively on natural language processing tools to extract features, and the noise accumulating during the entire process significantly affects the extraction results. In this study, an end-to-end, attention mechanism-based convolutional neural network (CNN) is proposed. First, the attention mechanism is added to the input layer for automatic detection of more subtle clues and learning of parts of sentences that are relevant to relation extraction. Second, the sentence is encoded based on the position feature and word feature, a piecewise CNN (PCNN) is used to extract sentence features and classify relationships, and finally a max-margin loss function with a higher efficiency is used on the network. The accuracy of this method when used on the New York Times dataset is 2.0% higher than that of the classical PCNN+MIL model, and 1.0% higher than that of the classical APCNN+D model. The experimental results therefore demonstrate excellent accuracy of the proposed model when compared with that of other baselinemodels.

参考文献

[1] Liu Q, Li Y, Duan H, et al. A survey of knowledge mapping construction techniques[J]. Journal of Computer Research and Development, 2016, 53(3): 582-600.
[2] Mintz M, Bills S, Snow R. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011.
[3] Zeng D, Liu K, Chen Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]// Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2015: 1753-1762.
[4] Santos C N D, Xiang B, Zhou B. Classifying relations by ranking with convolutional neuralnetworks[J]. Computer Science, 2015, 86: 132-137.
[5] Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]// The 25th International Conference on Computational Linguistics: Technical Papers. 2014: 2335-2344.
[6] Katiyar A, Cardie C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees[C]// Proceedings of the Meeting of the Association for Computational Linguistics. 2017: 917-928.
[7] Liu T, Wang K, Chang B, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]// Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2017: 1790-1795.
[8] Lin Y, Shen S, Liu Z, et al. Neural relation extraction with selective attention overinstances[C]// Proceedings of the Meeting of the Association for Computational Linguistics. 2016: 2124-2133.
[9] Ji G L, Liu K, He S Z, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]// Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. 2017: 3060-3066.
[10] Li M Y, Yang J. Open Chinese entity relationship extraction method based on dependency parsing[J]. Computer Engineering, 2016, 42(6): 201-207.
[11] Riedel S, Yao L, Mccallum A. Modeling relations and their mentions without labeledtext[C]// Machine Learning and Knowledge Discovery in Databases. 2010: 148-163.
[12] Hoffmann R, Zhang C, Ling X. Knowledge-based weak supervision for information extraction of overlapping relations[C]// Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011: 541-550.
[13] Surdeanu M, Tibshirani J, Nallapati R. Multi-instance multi-label learning for relation extraction[C]// Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 2012: 455-465.
[14] Finkel J R, Grenager T, Manning C. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling[C]// Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 2005: 363-370.
[15] Mintz M, Bills S, Snow R. Distant supervision for relation extraction without labeleddata[C]// Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011.
[16] Jiang X, Wang Q, Li P, et al. Relation extraction with multi-instance multi-label convolutional neural networks[C]// The 26th International Conference on Computational Linguistics: Technical Papers. 2016: 1471-1480.
[17] Yu X K, Chen L, Guo J, et al. Relationship extraction method combining clause-level remote supervision and semi-supervised integration learning[J]. Pattern Recognition and Artificial Intelligence, 2017, 30(1): 54-63.
[18] Jiao L C, Yang S Y, Liu F, et al. Neural network seventy years: retrospect and prospect[J]. Chinese Journal of Computers, 2016, 39(8): 1697-1716.
[19] Ren X, Wu Z, He W, et al. CoType: joint extraction of typed entities and relations with knowledge bases[C]// Proceedings of the 26th International Conference on World Wide Web. 2017: 1015-1024.
[20] Yang J F, Yu Q B, Guan Y, et al. A survey of research on electronic medical record named entity recognition and entity relationship extraction[J]. Acta Automatica Sinica, 2014, 40(8): 1537-1562.
[21] Wang L, Cao Z, Melo G D, et al. Relation classification via multi-level attention CNNs[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1298-1307.
文章导航

/