Towards Improving Neural Named Entity Recognition with Gazetteers

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5301–5307 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5301 Towards Improving Neural Named Entity Recognition with Gazetteers Tianyu Liu∗ Peking University ty-liu@pku.edu.cn Jin-Ge Yao Chin-Yew Lin Microsoft Research Asia {jinge.yao,cyl}@microsoft.com Abstract Most of the recently proposed neural models for named entity recognition have been purely data-driven, with a strong emphasis on get- ting rid of the efforts for collecting external resources or designing hand-crafted features. This could increase the chance of overfitting since the models cannot access any supervi- sion signal beyond the small amount of anno- tated data, limiting their power to generalize beyond the annotated entities. In this work, we show that properly utilizing external gazetteers could benefit segmental neural NER models. We add a simple module on the recently pro- posed hybrid semi-Markov CRF architecture and observe some promising results. 1 Introduction In the past few years, neural models have become dominant in research on named entity recognition (NER) (Lample et al., 2016; Ma and Hovy, 2016; Chiu and Nichols, 2016, inter alia), as they ef- fectively utilize distributed representations learned from large-scale unlabeled texts (Pennington et al., 2014; Peters et al., 2018; Devlin et al., 2018, inter alia), while avoiding the huge efforts required for designing hand-crafted features or gathering exter- nal lexicons. Results from modern neural NER models have achieved new state-of-the-art per- formance over standard benchmarks such as the popular CoNLL 2003 shared task dataset (Tjong Kim Sang and De Meulder, 2003). An end-to-end model with the property of let- ting the data speak for itself seems to be appeal- ing at first sight. However, given that the amount of labeled training data for NER is relatively small when compared with other tasks with millions of training examples, the annotated entities could only achieve a rather limited coverage for a the- oretically infinite space of variant entity names. ∗Work during internship at Microsoft Research Asia Moreover, current neural architectures heavily rely on the word form due to the use of word embed- dings and character embeddings, which could lead to a high chance of overfitting. 1 For instance, all the appearances of the single token Clinton in the CoNLL 2003 dataset are person names, while in practice it is also possible to refer to loca- tions.2 Data-driven end-to-end models trained on that dataset could implicitly bias towards predict- ing PERSON for most occurrences of Clinton even under some contexts when it refers to a location. On the other hand, for frequently studied lan- guages such as English, people have already collected dictionaries or lexicons consisting of long lists of entity names, known as gazetteers. Gazetteers could be treated as an external source of knowledge that could guide models towards wider coverage beyond the annotated entities in NER datasets. In traditional log-linear named en- tity taggers (Ratinov and Roth, 2009; Luo et al., 2015), gazetteers are commonly used as discrete features in the form of whether …

Content truncated to 500 words