Bidirectional LSTM-CRF for Named Entity Recognition

Bidirectional LSTM-CRF for Named Entity Recognition Rrubaa Panchendrarajan School of Computing National University of Singapore rrubaa@comp.nus.edu.sg Aravindh Amaresan School of Computing National University of Singapore e0146310@u.nus.edu Abstract Named Entity Recognition (NER) is a chal- lenging sequence labeling task which requires a deep understanding of the orthographic and distributional representation of words. In this paper, we propose a novel neural architec- ture that benefits from word and character level information and dependencies across ad- jacent labels. This model includes bidirec- tional LSTM (BI-LSTM) with a bidirectional Conditional Random Field (BI-CRF) layer. Our work is the first to experiment BI-CRF in neural architectures for sequence labeling task. We show that CRF can be extended to capture the dependencies between labels in both right and left directions of the sequence. This variation of CRF is referred to as BI-CRF and our results show that BI-CRF improves the performance of the NER model compare to an unidirectional CRF and backward CRF is capable of capturing most difficult entities compare to the forward CRF. Our system is competitive on the CoNLL-2003 dataset for English and outperforms most of the existing approaches which do not use any external la- beled data. 1 Introduction Named Entity Recognition is an important task in Natural Language Processing (NLP) which has drawn the attention for a few decades. NER is widely used in downstream applications of NLP and artificial intelligence such as machine trans- lation, information retrieval, and question answer- ing. Initially experimented sequence labeling mod- els are linear statistical models which include Hid- den Markov models (HMM) (Morwal et al., 2012), Maximum Entropy models (MEMM) (McCallum et al., 2000) and Conditional Random Fields (CRF) (Lafferty et al., 2001). Later, (Qi et al., 2009) proposed an effective neural network model using the Convolutional neural network (CNN) combined with CRF. However, this was purely based on word embedding and not capable of processing variable length input. A well-studied solution for a neural network to take into account an effectively infinite amount of context is the BI-LSTM. (Graves et al., 2013) suc- cessfully applied BI-LSTM for speech recognition task. CNNs have also been investigated for mod- eling character-level information, among other NLP tasks and combination of BI-LSTM, CNN and CRF has been shown to be very successful in the field of sequence labeling task in past few years. BI-LSTM- CRF (Huang et al., 2015), BI-LSTM-CNN (Chiu and Nichols, 2016), BI-LSTM-CRF (Lample et al., 2016) and LSTM-CNN-CRF (Ma and Hovy, 2016) are few such architectures. Most recent approaches to NER have focused on multi-task learning (Liu et al., 2018a) which jointly conduct other related NLP tasks like entity linking or chunking. Moreover, very complex architectures such as stacked Recur- rent Neural Network (Tran et al., 2017) and stack of classifiers (Liu et al., 2018b) are experimented in state-of-the-art models. Conditional random field (CRF) jointly models the label decision by capturing the dependencies across adjacent labels. For example, I-XXX (label in- dicates that the given word appears inside the name PACLIC 32 531 32nd …

Content truncated to 500 words