Gazetteer-Enhanced Attentive Neural Networks for Named Entity Recognition

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 6232–6237, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 6232 Gazetteer-Enhanced Attentive Neural Networks for Named Entity Recognition Hongyu Lin1,3, Yaojie Lu1,3, Xianpei Han1,2,∗, Le Sun1,2, Bin Dong4, Shanshan Jiang4 1Chinese Information Processing Laboratory 2State Key Laboratory of Computer Science Institute of Software, Chinese Academy of Sciences, Beijing, China 3University of Chinese Academy of Sciences, Beijing, China 4Ricoh Software Research Center (Beijing) Co.,Ltd. {hongyu2016,yaojie2017,xianpei,sunle}@iscas.ac.cn {bin.dong,shanshan.jiang}@srcb.ricoh.com Abstract Current region-based NER models only rely on fully-annotated training data to learn ef- fective region encoder, which often face the training data bottleneck. To alleviate this prob- lem, this paper proposes Gazetteer-Enhanced Attentive Neural Networks, which can en- hance region-based NER by learning name knowledge of entity mentions from easily- obtainable gazetteers, rather than only from fully-annotated data. Specially, we first pro- pose an attentive neural network (ANN), which explicitly models the mention-context associa- tion and therefore is convenient for integrating externally-learned knowledge. Then we de- sign an auxiliary gazetteer network, which can effectively encode name regularity of mention- s only using gazetteers. Finally, the learned gazetteer network is incorporated into ANN for better NER. Experiments show that our ANN can achieve the state-of-the-art perfor- mance on ACE2005 named entity recognition benchmark. Besides, incorporating gazetteer network can further improve the performance and significantly reduce the requirement of training data. 1 Introduction Named entity recognition (NER), aiming to iden- tify text mentions of specific entity types, is a fundamental NLP task. Recently, region-based NER approaches (Finkel and Manning, 2009; Xu et al., 2017; Sohrab and Miwa, 2018) have at- tracted significant attention, which first encode all candidate regions (commonly all subsequences of a sentence) using a region encoder, then identify whether each subsequence is an entity mention of target types using a classifier. For example, in Figure 1 all subsequences of the sentence, such as “George Washington”, will first be encoded, ∗Xianpei Han is the corresponding author. a Sentence Candidate Region Inner-Region Encoder Attentive Context Encoder Region Representation Utterance Encoder Utterance Utterance Representation … PER … PER ORG Easily Available Gazetteer [ George Washington ] was an leader. George Washington (a) Attentive Neural Network (b) Gazetteer Network Handcraft Training Data Figure 1: The overall architecture of GEANN. The can- didate region is “George Washington”, which literally could be a person or an organization (university). and then be classified into entity types. Com- pared to sequential labeling models, region-based models can naturally detect nested or overlapping mentions by considering all subsequences, and therefore are of great value to NER. Generally, an effective region encoder should capture two kinds of knowledge for NER. One is name knowledge, which encodes the inner compositional regularity of entity mentions, i.e., how likely a subsequence itself will be an entity mention. For example, a region encoder should know “George Washington” is a valid PER name because “George” is a common first name and “Washington” is …

Content truncated to 500 words