Pyramid: A Layered Model for Nested Named Entity Recognition

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5918–5928 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5918 Pyramid: A Layered Model for Nested Named Entity Recognition Jue Wang †, Lidan Shou ‡†∗, Ke Chen †, Gang Chen ‡† ‡State Key Laboratory of CAD&CG, †College of Computer Science and Technology, Zhejiang University, China {zjuwangjue,should,chenk,cg}@zju.edu.cn Abstract This paper presents Pyramid, a novel lay- ered model for Nested Named Entity Recog- nition (nested NER). In our approach, token or text region embeddings are recursively in- putted into L flat NER layers, from bottom to top, stacked in a pyramid shape. Each time an embedding passes through a layer of the pyra- mid, its length is reduced by one. Its hidden state at layer l represents an l-gram in the input text, which is labeled only if its corresponding text region represents a complete entity men- tion. We also design an inverse pyramid to allow bidirectional interaction between layers. The proposed method achieves state-of-the-art F1 scores in nested NER on ACE-2004, ACE- 2005, GENIA, and NNE, which are 80.27, 79.42, 77.78, and 93.70 with conventional em- beddings, and 87.74, 86.34, 79.31, and 94.68 with pre-trained contextualized embeddings. In addition, our model can be used for the more general task of Overlapping Named En- tity Recognition. A preliminary experiment confirms the effectiveness of our method in overlapping NER. 1 Introduction Named Entity Recognition (NER), which aims at identifying text spans as well as their semantic classes, is an essential and fundamental Natural Language Processing (NLP) task. It is typically modeled as a sequence labeling problem, which can be effectively solved by RNN-based approach (Huang et al., 2015; Lample et al., 2016; Ma and Hovy, 2016). However, such formulation oversim- plifies the problem and is based on a very strong as- sumption that entity mentions do not overlap with each other, which is certainly not the real case. In real-world languages, entities might be deeply nested or overlapping, calling for better models to handle such complexity. ∗Corresponding author Former U.N. Ambassador Jeane Kirkpatrick ... Former�U.N. U.N. Ambassador Ambassador Jeane Jeane Kirkpatrick ... Former�U.N. Ambassador U.N.�Ambassador Jeane Ambassador Jeane�Kirkpatrick ... Former�U.N. Ambassador�Jeane U.N. Ambassador Jeane Kirkpatrick ... Former�U.N.�Ambassador Jeane Kirkpatrick ... layer�1 layer�2 layer�3 layer�4 layer�5 inputs:Former U.N. Ambassador Jeane Kirkpatrick ... labels: ORG ROLE FIRST NAME ROLE PER ROLE PER Figure 1: Pyramid output of a sentence from NNE (Ringland et al., 2019) containing 8 nested entities. Many previous studies have focused on recog- nizing nested entity mentions. A few works use proprietary structures, such as constituency graph (Finkel and Manning, 2009) or hypergraph (Lu and Roth, 2015; Muis and Lu, 2017), to explicitly cap- ture nested entities. These structures, however, do not produce satisfactory performance results. Some other works handle nested entity mentions in a layered model, which employs multiple flat NER layers(Alex et al., 2007; Ju et al., 2018; Fisher and Vlachos, 2019). Each layer is usually responsi- ble for predicting a group of nested entities …

Content truncated to 500 words