SeqL at SemEval-2022 Task 11: An Ensemble of Transformer Based Models for Complex Named Entity Recognition Task

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1583 - 1592 July 14-15, 2022 ©2022 Association for Computational Linguistics SeqL at SemEval-2022 Task 11: An Ensemble of Transformer Based Models for Complex Named Entity Recognition Task Fadi Hassan1, Wondimagegnhue Tufa2, Guillem Collell1, Piek Vossen2, Lisa Beinborn2, Adrian Flanagan1, and Kuan Eeik Tan1 1Helsinki Research Center, Europe Cloud Service Competence Center Huawei Technologies Oy (Finland) Co. Ltd., Helsinki, Finland {firstname.lastname}@huawei.com 2Faculty of Humanities, Vrije Universiteit Amsterdam {w.t.tufa, p.t.j.m.vossen, l.beinborn}@vu.nl Abstract In this paper, we present a system for detect- ing complex named entities in multilingual and code-mix settings. We discuss the results obtained in task 11 (MultiCoNER) of the Se- mEval 2022 competition. The model is an en- semble of various transformer-based language models combined with a Conditional Random Field (CRF) layer. Our model ranks fourth in track 12 (multilingual track) and fifth in track 13 (code-mixed track). We describe the de- tails of our model implementation and discuss the effect of different aggregation methods. Fi- nally, we conduct additional analyses to under- stand the performance differences between lan- guages. 1 Introduction Named Entity Recognition (NER) is the task of identifying proper names in a text and categorizing them into predefined entity types such as person (PER), location (LOC), or creative work (CW). For example, given a sentence "Michael Jeffrey Jordan was born in Brooklyn, New York" the goal is to label entities correctly with their correspond- ing category using the BIO-scheme: "Michael [B-PER], Jeffrey [I-PER], Jor- dan [I-PER] was born in Brooklyn [B- LOC], New [I-LOC] York [I-LOC]" Existing systems are often trained on standard news text and strongly rely on surface form features such as capitalization and punctuation. These approaches do not scale well to user- generated content because of the increased varia- tion in language and context in an ever-expanding domain (Meng et al., 2021; Fetahu et al., 2021; Augenstein et al., 2017). Examples of challenging scenarios for named entity recognition are: (a) Entities in very short text inputs such as search queries with limited or no context (Meng et al., 2021) (b) Structurally complex entities such as movie or book titles rang- ing from complex noun phrases to full clauses (c) Recognizing named entities in dynamically evolv- ing contexts in which novel entities emerge (Au- genstein et al., 2017; Aguilar et al., 2019). Although many named entities are shared be- tween languages, named entity recognition sys- tems usually rely on language-specific cues (e.g. capitalization of nouns in German, compound- ing phenomena in agglutinative languages such as Korean, Japanese and Turkish (Agerri and Rigau, 2016)). Such kind of detection cues do not scale well to other languages. As a conse- quence, most models need to be fine-tuned for each language separately on manually annotated high-quality training data which is a costly pro- cess. Task 12 of SemEval 2022 provides a test bench for more robust systems which can detect complex named entities in 11 languages (Malmasi et al., 2022b). The dataset intentionally contains seman- tically …

Content truncated to 500 words