IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition Using Knowledge Bases

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1335–1346 July 13-14, 2023 ©2023 Association for Computational Linguistics IXA/Cogcomp at SemEval-2023 Task 2: Context-enriched Multilingual Named Entity Recognition using Knowledge Bases Iker García-Ferrero1, Jon Ander Campos1, Oscar Sainz1, Ander Salaberria1, Dan Roth2 1HiTZ Center - Ixa, University of the Basque Country UPV/EHU {iker.garciaf,jonander.campos,oscar.sainz,ander.salaberria}@ehu.eus 2University of Pennsylvania danroth@seas.upenn.edu Abstract Named Entity Recognition (NER) is a core natural language processing task in which pre- trained language models have shown remark- able performance. However, standard bench- marks like CoNLL 2003 (Tjong Kim Sang and De Meulder, 2003) do not address many of the challenges that deployed NER systems face, such as having to classify emerging or com- plex entities in a fine-grained way. In this pa- per we present a novel NER cascade approach comprising three steps: first, identifying can- didate entities in the input sentence; second, linking the each candidate to an existing knowl- edge base; third, predicting the fine-grained category for each entity candidate. We empiri- cally demonstrate the significance of external knowledge bases in accurately classifying fine- grained and emerging entities. Our system ex- hibits robust performance in the MultiCoNER2 (Fetahu et al., 2023b) shared task, even in the low-resource language setting where we leverage knowledge bases of high-resource lan- guages. 1 Introduction The research on the Named Entity Recognition field has mainly focused on recognizing enti- ties from news corpora. Datasets like CoNLL 2003 (Tjong Kim Sang and De Meulder, 2003) or OntoNotes (Hovy et al., 2006) have been the stan- dard benchmarks for a long time. It is not difficult to find systems that perform over 92% of F1 score on those datasets. However, it is known that actual models perform pretty well on relatively easy entity types (such as person names) but have been strug- gling to extract more complex entities (Luken et al., 2018; Hanselowski et al., 2018). Moreover, the evaluation setup usually involves a large overlap on the entities from the train and test sets, over- estimating the systems performance. Removing such overlap resulted on a large performance drop, Figure 1: Illustration of our fine-grained named entity recognition system. consequently showing that models tend to memo- rize rather than generalize to complex or unseen entities (Meng et al., 2021; Fetahu et al., 2022). Motivated by these insights, recent works have developed new and more complex datasets like MultiCoNER (Malmasi et al., 2022a) or UFET (Choi et al., 2018) that involve complex entity mentions with higher granularity on the type 1335 definition. To address these new challenges large language models (LLMs) are commonly used, with the hope that after being trained on billions of to- kens the models become aware of the relevant enti- ties. Despite the capacity of the models to memo- rize information about each individual entity, they face a harsh reality: the knowledge that a model can memorize is limited by the date on which the model was pre-trained. So, new emerging entities can be difficult to categorize for models …

Content truncated to 500 words