A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition

Proceedings of Recent Advances in Natural Language Processing, pages 1104–1113, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_127 1104 A Morpho-Syntactically Informed LSTM-CRF Model for Named Entity Recognition Lilia Simeonova FMI Sofia University Bulgaria lilia.valentinova. simeonova @gmail.com Kiril Simov, Petya Osenova LMaKP IICT-BAS Bulgaria {kivs,petya} @bultreebank.org Preslav Nakov Qatar Computing Research Institute, HBKU Qatar pnakov@qf.org.qa Abstract We propose a morphologically informed model for named entity recognition, which is based on LSTM-CRF archi- tecture and combines word embeddings, Bi-LSTM character embeddings, part-of- speech (POS) tags, and morphological information. While previous work has focused on learning from raw word in- put, using word and character embeddings only, we show that for morphologically rich languages, such as Bulgarian, access to POS information contributes more to the performance gains than the detailed morphological information. Thus, we show that named entity recognition needs only coarse-grained POS tags, but at the same time it can benefit from simulta- neously using some POS information of different granularity. Our evaluation re- sults over a standard dataset show sizeable improvements over the state-of-the-art for Bulgarian NER. 1 Introduction Although in recent years the Named Entity Link- ing (also known as Named Entity Disambiguation) task has been central in NLP research, the Named- entity recognition (NER) task has remained far from solve, having in mind the productivity of names and the amount of information available in the era of big and noisy data. NER plays a critical role in the processing of texts with application to many real-world Natural Language Processing (NLP) tasks such as Ques- tion Answering, Information Extraction, Machine Translation, Dialog Systems, and chatbots, where it is sometimes called Concept Segmentation and Labeling (Saleh et al., 2014). Traditionally, NER has focused on recognizing en- tities such as person (PER), organization (ORG), location (LOC), and miscellaneous (MISC). This tradition goes back to the Message Understand- ing Conference (MUC) for English (Grishman and Sundheim, 1996), and the subsequent CoNLL 2002/2003 shared tasks, which also targeted other European Languages such as Spanish, Dutch, and German (Tjong Kim Sang and De Meulder, 2003).1 This same setup was followed in more re- cent work for a number of other languages, and we also follow it in the present work. Early systems relied on hand-crafted rules with pattern-matching (Appelt et al., 1995). Un- fortunately, this required an large pre-annotated datasets, collecting which was time-consuming and error-prone. The next step was to add gazetteers and lexicons that were generated auto- matically or semi-automatically (Popescu and Et- zioni, 2005). Adding such resources required spe- cial approaches to resolve the ambiguity between names and common words. Such problems were solved using models such as Hidden Markov Mod- els (Zhou and Su, 2002) and Conditional Random Fields (Sutton and McCallum, 2012). In our work here, we use deep neural net- works for Bulgarian NER. Lample et al. (2016) have shown remarkable results for English, us- ing a combination of Bi-LSTMs (Bi-directional Long Short-Term Memory) and CRF. However, the approach is problematic for morphologically rich languages. The main problem is the …

Content truncated to 500 words