Neural Architectures for Nested NER through Linearization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5326–5331 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5326 Neural Architectures for Nested NER through Linearization Jana Strakov´a and Milan Straka and Jan Hajiˇc Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics {strakova,straka,hajic}@ufal.mff.cuni.cz Abstract We propose two neural network architectures for nested named entity recognition (NER), a setting in which named entities may overlap and also be labeled with more than one label. We encode the nested labels using a linearized scheme. In our first proposed approach, the nested labels are modeled as multilabels cor- responding to the Cartesian product of the nested labels in a standard LSTM-CRF archi- tecture. In the second one, the nested NER is viewed as a sequence-to-sequence problem, in which the input sequence consists of the to- kens and output sequence of the labels, using hard attention on the word whose label is be- ing predicted. The proposed methods outper- form the nested NER state of the art on four corpora: ACE-2004, ACE-2005, GENIA and Czech CNEC. We also enrich our architectures with the recently published contextual embed- dings: ELMo, BERT and Flair, reaching fur- ther improvements for the four nested entity corpora. In addition, we report flat NER state- of-the-art results for CoNLL-2002 Dutch and Spanish and for CoNLL-2003 English. 1 Introduction In nested named entity recognition, entities can be overlapping and labeled with more than one la- bel such as in the example “The Florida Supreme Court” containing two overlapping named entities “The Florida Supreme Court” and “Florida”.1 Recent publications on nested named entity recognition involve stacked LSTM-CRF NE rec- ognizer (Ju et al., 2018), or a construction of a special structure that explicitly captures the nested entities, such as a constituency graph (Finkel and Manning, 2009) or various modifications of a di- rected hypergraph (Lu and Roth, 2015; Katiyar and Cardie, 2018; Wang and Lu, 2018). 1Example from ACE-2004 (Doddington et al., 2004), https://catalog.ldc.upenn.edu/LDC2005T09. We propose two completely neural network ar- chitectures for nested nested named entity recog- nition which do not explicitly build or model any structure and infer the relationships between nested NEs implicitly: • In the first model, we concatenate the nested entity multiple labels into one multilabel, which is then predicted with a standard LSTM-CRF (Lample et al., 2016) model. The advantages of this model are simplicity and effectiveness, because an already exist- ing NE pipeline can be reused to model the nested entities. The obvious disadvantage is a large growth of NE classes. • In the second model, the nested entities are encoded in a sequence and then the task can be viewed as a sequence-to-sequence (seq2seq) task, in which the input sequence are the tokens (forms) and the output se- quence are the labels. The decoder predicts labels for each token, until a special label "<eow>" (end of word) is predicted and the decoder moves to the next token. The expressiveness of …

Content truncated to 500 words