Neural Modeling for Named Entities and Morphology (NEMO2)

Neural Modeling for Named Entities and Morphology (NEMO2) Dan Bareket1,2 and Reut Tsarfaty1 1Bar Ilan University, Ramat-Gan, Israel 2Open Media and Information Lab (OMILab), The Open University of Israel, Israel dbareket@gmail.com, reut.tsarfaty@biu.ac.il Abstract Named Entity Recognition (NER) is a funda- mental NLP task, commonly formulated as classification over a sequence of tokens. Mor- phologically rich languages (MRLs) pose a challenge to this basic formulation, as the boundaries of named entities do not neces- sarily coincide with token boundaries, rather, they respect morphological boundaries. To ad- dress NER in MRLs we then need to answer two fundamental questions, namely, what are the basic units to be labeled, and how can these units be detected and classified in realistic set- tings (i.e., where no gold morphology is avail- able). We empirically investigate these questions on a novel NER benchmark, with parallel token- level and morpheme-level NER annotations, which we develop for Modern Hebrew, a mor- phologically rich-and-ambiguous language. Our results show that explicitly modeling morphol- ogical boundaries leads to improved NER per- formance, and that a novel hybrid architecture, in which NER precedes and prunes morpho- logical decomposition, greatly outperforms the standard pipeline, where morphological de- composition strictly precedes NER, setting a new performance bar for both Hebrew NER and Hebrew morphological decomposition tasks. 1 Introduction Named Entity Recognition (NER) is a fundamental task in the area of Information Extraction (IE), in which mentions of Named Entities (NE) are extracted and classified in naturally occurring texts. This task is most commonly formulated as a sequence labeling task, where extraction takes the form of assigning each input token with a label that marks the boundaries of the NE (e.g., B, I, O), and classification takes the form of assigning labels to indicate entity type (PER, ORG, LOC, etc.). Despite a common initial impression from latest NER performance, brought about by neural models on the main English NER benchmarks— CoNLL 2003 (Tjong Kim Sang, 2003) and Onto- Notes (Weischedel et al., 2013)—the NER task in real-world settings is far from solved. Spe- cifically, NER performance is shown to greatly diminish when moving to other domains (Luan et al., 2018; Song et al., 2018), when address- ing the long tail of rare, unseen, and new user-generated entities (Derczynski et al., 2017), and when handling languages with fundamentally different structure than English. In particular, there is no readily available and empirically verified neural modeling strategy for neural NER in those languages with complex word-internal structure, also known as morphologically rich languages. Morphologically rich languages (MRL) (Tsarfaty et al., 2010; Seddah et al., 2013; Tsarfaty et al., 2020) are languages in which substantial information concerning the arrangement of words into phrases and the relations between them is expressed at the word level, rather than in a fixed word-order or a rigid structure. The extended amount of information expressed at word-level and the morpho-phonological processes creating these words result in high token-internal com- plexity, which poses serious challenges to the ba- sic formulation of NER as classification of …

Content truncated to 500 words