Named Entity Recognition with Bidirectional LSTM-CNNs

Named Entity Recognition with Bidirectional LSTM-CNNs Jason P.C. Chiu University of British Columbia jsonchiu@gmail.com Eric Nichols Honda Research Institute Japan Co.,Ltd. e.nichols@jp.honda-ri.com Abstract Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineer- ing and lexicons to achieve high performance. In this paper, we present a novel neural net- work architecture that automatically detects word- and character-level features using a hy- brid bidirectional LSTM and CNN architec- ture, eliminating the need for most feature en- gineering. We also propose a novel method of encoding partial lexicon matches in neu- ral networks and compare it to existing ap- proaches. Extensive evaluation shows that, given only tokenized text and publicly avail- able word embeddings, our system is com- petitive on the CoNLL-2003 dataset and sur- passes the previously reported state of the art performance on the OntoNotes 5.0 dataset by 2.13 F1 points. By using two lexicons con- structed from publicly-available sources, we establish new state of the art performance with an F1 score of 91.62 on CoNLL-2003 and 86.28 on OntoNotes, surpassing systems that employ heavy feature engineering, proprietary lexicons, and rich entity linking information. 1 Introduction Named entity recognition is an important task in NLP. High performance approaches have been dom- inated by applying CRF, SVM, or perceptron models to hand-crafted features (Ratinov and Roth, 2009; Passos et al., 2014; Luo et al., 2015). However, Collobert et al. (2011b) proposed an effective neu- ral network model that requires little feature engi- neering and instead learns important features from word embeddings trained on large quantities of un- labelled text – an approach made possible by recent advancements in unsupervised learning of word em- beddings on massive amounts of data (Collobert and Weston, 2008; Mikolov et al., 2013) and neural net- work training algorithms permitting deep architec- tures (Rumelhart et al., 1986). Unfortunately there are many limitations to the model proposed by Collobert et al. (2011b). First, it uses a simple feed-forward neural network, which restricts the use of context to a fixed sized window around each word – an approach that discards use- ful long-distance relations between words. Second, by depending solely on word embeddings, it is un- able to exploit explicit character level features such as prefix and suffix, which could be useful especially with rare words where word embeddings are poorly trained. We seek to address these issues by propos- ing a more powerful neural network model. A well-studied solution for a neural network to process variable length input and have long term memory is the recurrent neural network (RNN) (Goller and Kuchler, 1996). Recently, RNNs have shown great success in diverse NLP tasks such as speech recognition (Graves et al., 2013), machine translation (Cho et al., 2014), and language mod- eling (Mikolov et al., 2011). The long-short term memory (LSTM) unit with the forget gate allows highly non-trivial long-distance dependencies to be easily learned (Gers et al., 2000). For sequential la- belling tasks such as NER and …

Content truncated to 500 words