结合上下文词汇匹配和图卷积的材料数据命名实体识别

doi:10.12066/j.issn.1007-2861.2377

Abstract

Abstract:

Literature pertaining to materials contain abundant information regarding data mining using machine learning and natural language processing, which is currently being investigated extensively. Named entity recognition (NER) is first performed when mining and extracting information from data such that the data can be used efficiently. As vector representation cannot solve multiple meanings of words, and models often extract contextual features while disregarding global features, a named entity recognition method based on matching contextual lexical words and graph convolution is proposed herein. First, the contextual dynamic features of text is obtained using XLNet; second, the contextual and global features are obtained using a long short-term memory network and a graph convolutional network (GCN) combined with contextual lexical words of the text, respectively. Finally, a sequence of labels is output via a conditional random field. The model is validated using two different datasets. Experimental results of the material data show that the precision, recall, and F1 score are 90.05%, 88.67%, and 89.36%, respectively, which effectively improve the named entity recognition accuracy.

Key words: named entity recognition (NER), XLNet, graph convolutional network (GCN)

CLC Number:

TP 391

CHEN Qian, WU Xing. Material data named entity recognition based on matching contextual lexical words and graph convolution[J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 372-385.

Figures/Tables 10

Fig.1

Fig.2

Fig.3

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Table 7

References 31

[1]	Wang W R, Jiang X, Tian S H, et al. Automated pipeline for superalloy data by text mining[EB/OL]. [2022-01-13]. https://xueshu.baidu.com/usercenter/paper/show?paperid=1t1308h0f93g0xk0d73p0xs0bb071138.
[2]	Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[J]. Advances in Neural Information Processing Systems, 2013, 26: 3111-3120.
[3]	Pennington J, Socher R, Manning C. Glove: global vectors for word representation[C]// Conference on Empirical Methods in Natural Language Processing. 2014: 1532-1543.
[4]	Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. pmid: 9377276
[5]	LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code[EB/OL]. [2021-11-30]. https://xueshu.baidu.com/usercenter/paper/show?paperid=3340934c7fc4377155350acaf8632c64&site=xueshu_se&hitarticle=1&sc_from=nuc.
[6]	Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019: 4171-4186.
[7]	Liu Y, Ott M, Goyal N, et al. RoBERTa: a robustly optimized bert pretraining approach[DB/OL]. [2021-12-30]. https://arxiv.org/abs/1907.11692v1.
[8]	Yang Z L, Dai Z H, Yang Y M, et al. XLNet: generalized autoregressive pretraining for language understanding[DB/OL]. [2022-01-05]. https://arxiv.org/abs/1906.08237.
[9]	Grishman R. The NYU system for MUC-6 or where's the syntax?[C]// Proceedings of the 6th Conference on Messgae Understanding. 1995: 167-175.
[10]	Zhang Y, Yang J. Chinese NER using lattice LSTM[C]// The 56th Annual Meeting of the Association for Computational Linguistics (ACL). 2018: 1554-1564.
[11]	Wu F Z, Liu J X, Wu C H, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and Joint training with word segmentation[C]// The World Wide Web Conference. 2019: 3342-3348.
[12]	武惠, 吕立, 于碧辉. 基于迁移学习和 BiLSTM-CRF 的中文命名实体识别[J]. 小型微型计算机系统, 2019, 40(6): 1142-1147.
[13]	王红斌, 沈强, 线岩团. 融合迁移学习的中文命名实体识别[J]. 小型微型计算机系统, 2017, 38(2): 346-351.
[14]	王银瑞, 彭敦陆, 陈章, 等. Trans-NER: 一种迁移学习支持下的中文命名实体识别模型[J]. 小型微型计算机系统, 2019, 40(8): 1622-1626.
[15]	王栋, 李业刚, 张晓, 等. 基于准循环神经网络的中文命名实体识别[J]. 计算机工程与设计, 2020, 41(7): 2038-2043.
[16]	Huang Z, Wei X, Kai Y. Bidirectional LSTM-CRF models for sequence tagging[EB/OL]. [2022-01-05]. https://xueshu.baidu.com/usercenter/paper/show?paperid=ee0eabacde06f546fe80624e0084647b.
[17]	Peng M L, Xing X Y, Zhang Q, et al. Distantly supervised named entity recognition using positive-unlabeled learning[DB/OL]. [2022-01-17]. https://arxiv.org/abs/1906.01378.
[18]	Liu Y J, Meng F D, Zhang J C, et al. GCDT: a global context enhanced deep transition architecture for sequence labeling[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 2431-2441.
[19]	Lison P, Hubin A, Barnes J, et al. Named entity recognition without labelled data: a weak supervision approach[DB/OL]. [2022-01-18]. https://arxiv.org/abs/2004.14723.
[20]	Lin B Y, Lee D H, Shen M, et al. TriggerNER: learning with entity triggers as explanations for named entity recognition[EB/OL]. [2021-12-05]. https://blog.csdn.net/comeonfly666/article/details/115421428.
[21]	Dai Z, Yang Z L, Yang Y M, et al. Transformer-XL: attentive language models beyond a fixed-length context[DB/OL]. [2021-12-28]. https://arxiv.org/abs/1901.02860v3.
[22]	Kip F T N, Welling M. Semi-supervised classification with graph convolutional networks[DB/OL]. [2021-12-01]. https://arxiv.org/abs/1609.02907.
[23]	Sui D, Chen Y, Liu K, et al. Leverage lexical knowledge for chinese named entity recognition via collaborative graph network[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019: 3828-3838.
[24]	Lee L H, Lu Y. Multiple embeddings enhanced multi-graph neural networks for chinese healthcare named entity recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2801-2810. doi: 10.1109/JBHI.2020.3048700
[25]	Sang E F T K, de Meulder F. Introduction to the CoNLL-2003 shared task: language-independent named entity recognition[C]// Proceedings of the Seventh Conforence on Natual Language Learning at HLT-NAACL. 2003: 142-147.
[26]	Weston L, Tshitoyan V, Dagdelen J, et al. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature[EB/OL]. [2021-11-28]. https://xueshu.baidu.com/usercenter/paper/show?paperid=186e0410a6000eb0bh5f0ck0x0214874&site=xueshu_se.
[27]	Tshitoyan V, Dagdelen J, Weston L, et al. Unsupervised word embeddings capture latent knowledge from materials science literature[J]. Nature, 2019, 571(7763): 95-98. doi: 10.1038/s41586-019-1335-8
[28]	Ramshaw L A, Marcus M P. Text chunking using transformation-based learning[EB/OL]. [2022-01-20]. https://xueshu.baidu.com/usercenter/paper/show?paperid=16cd0df4ac1924fb58369d003b87acd.
[29]	Chiu J, Nichols E. Named entity recognition with bidirectional LSTM-CNNs[EB/OL]. [2021-10-30]. https://xueshu.baidu.com/usercenter/paper/show?paperid=216cd0df4ac1924fb58369d003b87acd&site=xueshu_se.
[30]	Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2018: 2227-2237.
[31]	Martins P H, Marinho Z, Martins A. Joint learning of named entity recognition and entity linking[EB/OL]. [2022-01-24]. https://xueshu.baidu.com/usercenter/paper/show?paperid=1h4m025071620ag0hp0m08j0f8096616.

Material data named entity recognition based on matching contextual lexical words and graph convolution

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 31

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	YUE Xichao, FENG Yan, LIU Jian, YU Yeyong, XI Kangjie, QIAN Quan. Database for materials genome engineering [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 399-412.
[2]	XU Yuqin, QIAN Quan. Blockchain based data copyright protection and combinatorial auction [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 413-426.
[3]	LIU Fen, FAN Hongqiang, LÜ Tao, LI Qian, QIAN Quan. Kalman filter based method for processing small noisy sample data [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 427-439.
[4]	YAO Yu, HU Tao, FU Jianxun, HU Shunbo. Regression modeling and multi-objective optimization for small sample scattered data [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 451-462.
[5]	ZHANG Peng, ZHANG Rui. Feature selection based on reinforcement learning and its application in material informatics [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 463-475.
[6]	MAI Jiaqi, XU Pengcheng, DING Song, SUN Yangting, LU Wencong. Prediction of pitting potential for stainless steel by support vector regression [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 485-491.
[7]	CHEN Shuizhou, WANG Xiaoshu, OUYANG Qiubao, ZHANG Rui. Data-driven based properties prediction and reverse design of aluminum matrix composites [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(3): 512-522.
[8]	ZHANG Jin-yi1,2,3, XU De-zheng2, LI Ruo-han1,CHEN Xing-xiu1, XU Qin-le2. 9-axis MEMS-IMU real-time data fusion algorithm for attitude estimation [J]. Journal of Shanghai University（Natural Science Edition）, 2015, 21(5): 547-559.
[9]	SHEN Wen-feng, LI Feng-yong, ZHANG Xin-peng, YANG Qian-xing. Steganographic algorithm with histogram-preserving property [J]. Journal of Shanghai University（Natural Science Edition）, 2015, 21(2): 190-196.
[10]	YANG Qian-xing, LI Feng-yong, ZHANG Xin-peng, SHEN Wen-feng. Adaptive Robust Reversible Data Hiding Based on Modification of DCT Coefficient [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(5): 605-611.
[11]	ZHANG Qi1, HUANG Chun-chun1, HAN Hong2, LI Chao-lun2, WANG Wen-ping2. CEUS Image Segmentation of Carotid Arteries Using Multi-scale Fuzzy Clustering and DGVF Model [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(5): 633-644.
[12]	SHI Juan1, WANG De-yu1,2, TANG Gang1. Algorithm for Processing Optically Captured Motion Data [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(4): 489-497.
[13]	LI Ping1, YANG Ning1, ZHONG Shao-shan2. An Integrated Indoor Location Technology Based on RFID and DR [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(4): 498-504.
[14]	HOU Li-min, WU Wei-qi, ZHANG Xin-peng. Audio Re-sampling Detection in Audio Forensics Based on Second-Order Derivative [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(3): 304-312.
[15]	HE Xiang, LUO Xiang-feng. Document Clustering Method Based on Association Link Network [J]. Journal of Shanghai University（Natural Science Edition）, 2014, 20(2): 190-198.