Merge and Label: A Novel Neural Network Architecture for Nested NER

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5840–5850 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5840 Merge and Label: A novel neural network architecture for nested NER Joseph Fisher Department of Economics University of Cambridge jhjf2@cam.ac.uk Andreas Vlachos Dept. of Computer Science and Technology University of Cambridge andreas.vlachos@cst.cam.ac.uk Abstract Named entity recognition (NER) is one of the best studied tasks in natural language process- ing. However, most approaches are not ca- pable of handling nested structures which are common in many applications. In this paper we introduce a novel neural network architec- ture that first merges tokens and/or entities into entities forming nested structures, and then la- bels each of them independently. Unlike pre- vious work, our merge and label approach predicts real-valued instead of discrete seg- mentation structures, which allow it to com- bine word and nested entity embeddings while maintaining differentiability. We evaluate our approach using the ACE 2005 Corpus, where it achieves state-of-the-art F1 of 74.6, further improved with contextual embeddings (BERT) to 82.4, an overall improvement of close to 8 F1 points over previous approaches trained on the same data. Additionally we compare it against BiLSTM-CRFs, the dominant ap- proach for flat NER structures, demonstrating that its ability to predict nested structures does not impact performance in simpler cases.1 1 Introduction The task of nested named entity recognition (NER) focuses on recognizing and classifying en- tities that can be nested within each other, such as “United Kingdom” and “The Prime Minister of the United Kingdom” in Figure 1. Such entity struc- tures, while very commonly occurring, cannot be handled by the predominant variant of NER mod- els (McCallum and Li, 2003; Lample et al., 2016), which can only tag non-overlapping entities. A number of approaches have been proposed for nested NER. Lu and Roth (2015) introduced a hypergraph representation which can represent 1Code available at https://github.com/ fishjh2/merge_label overlapping mentions, which was further im- proved by Muis and Lu (2017), by assigning tags between each pair of consecutive words, prevent- ing the model from learning spurious structures (overlapping entity structures which are gramat- ically impossible). More recently, Katiyar and Cardie (2018) built on this approach, adapting an LSTM (Hochreiter and Schmidhuber, 1997) to learn the hypergraph directly, and Wang and Lu (2018) introduced a segmental hypergraph ap- proach, which is able to incorporate a larger num- ber of span based features, by encoding each span with an LSTM. Our approach decomposes nested NER into two stages. First tokens are merged into entities (Level 1 in Figure 1), which are merged with other to- kens or entities in higher levels. These merges are encoded as real-valued decisions, which enables a parameterized combination of word embeddings into entity embeddings at different levels. These entity embeddings are used to label the entities identified. The model itself consists of feedfor- ward neural network layers and is fully differen- tiable, thus it is straightforward to train with back- propagation. Unlike …

Content truncated to 500 words