Robust Lexical Features for Improved Neural Network Named-Entity Recognition

Proceedings of the 27th International Conference on Computational Linguistics, pages 1896–1907 Santa Fe, New Mexico, USA, August 20-26, 2018. 1896 Robust Lexical Features for Improved Neural Network Named-Entity Recognition Abbas Ghaddar RALI-DIRO Universit´e de Montr´eal Montr´eal, Canada abbas.ghaddar@umontreal.ca Philippe Langlais RALI-DIRO Universit´e de Montr´eal Montr´eal, Canada felipe@iro.umontreal.ca Abstract Neural network approaches to Named-Entity Recognition reduce the need for carefully hand- crafted features. While some features do remain in state-of-the-art systems, lexical features have been mostly discarded, with the exception of gazetteers. In this work, we show that this is unfair: lexical features are actually quite useful. We propose to embed words and entity types into a low- dimensional vector space we train from annotated data produced by distant supervision thanks to Wikipedia. From this, we compute — offline — a feature vector representing each word. When used with a vanilla recurrent neural network model, this representation yields substantial improvements. We establish a new state-of-the-art F1 score of 87.95 on ONTONOTES 5.0, while matching state-of-the-art performance with a F1 score of 91.73 on the over-studied CONLL-2003 dataset. 1 Introduction Named-Entity Recognition (NER) is the task of identifying textual mentions and classifying them into a predefined set of types. Various approaches have been proposed to tackle the task, from hand-crafted feature-based machine learning models like conditional random fields (Finkel et al., 2005) and percep- tron (Ratinov and Roth, 2009), to deep neural models (Collobert et al., 2011; Ma and Hovy, 2016; Strubell et al., 2017). Word representations (Turian et al., 2010; Mikolov et al., 2013), also known as word embeddings, are a key element for multiple NLP tasks including NER (Collobert et al., 2011). Due to the small amount of named-entity annotated data, embeddings are used to extend, rather than replace, hand-crafted features in order to obtain state-of-the-art performance (Lample et al., 2016). Recent studies (Yang et al., 2017; Søgaard and Goldberg, 2016) have explored methods for supplying deep sequential taggers with com- plementary features to standard embeddings. Peters et al. (2017) and Tran et al. (2017) tested special embeddings extracted from a neural language model (LM) trained on a large corpus. LM embeddings capture context-dependent aspects of word meaning using future (forward LM) and previous (backward LM) context words. When this information is added to standard features, it leads to significant im- provements in NER. Also, Chiu and Nichols (2016) showed that external knowledge resources (namely gazetteers) are crucial to NER performance. Gazetteer features encode the presence of word n-grams in predefined lists of NEs. In this work, we discuss some of the limitations of gazetteer features and propose an alternative lexical representation which is trained offline and that can be added to any neural NER system. In a nutshell, we embed words and entity types into a joint vector space by leveraging WiFiNE (Ghaddar and Langlais, 2018), a ressource which automatically annotates mentions in Wikipedia with 120 entity types. From this vector space, we compute for each word a 120-dimensional vector, where each dimension encodes the similarity of the word …

Content truncated to 500 words