Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 1127–1130 July 13-14, 2023 ©2023 Association for Computational Linguistics Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data Antonia Höfer and Mina Mottahedin Eberhard Karls Universität Tübingen Geschwister-Scholl-Platz 72074 Tübingen antonia-roswitha-sophie.hoefer@student.uni-tuebingen.de mina.mottahedin@student.uni-tuebingen.de Abstract Within the scope of the shared task Multi- CoNER II our aim was to improve the recogni- tion of named entities in English. We as team Minanto fine-tuned a cross-lingual model for Named Entity Recognition on English data and achieved an average F1 score of 51.47% in the final submission. We found that a monolin- gual model works better on English data than a cross-lingual and that the input of external data from earlier Named Entity Recognition tasks provides only minor improvements. In this pa- per we present our system, discuss our results and analyze the impact of external data. 1 Introduction Nowadays, language models are capable of solving various problems, amongst others grammar check- ing, speech (pattern) recognition and translation. But there are still some challenges. The recog- nition of complex named entities is one of them. Named entities (NEs) occur in various domains and can be, for instance, the name of a person, a city or the title of a book or a movie. As these NEs can be ambiguous like “On the Beach”, which can either be a preposition or a movie title, their recognition is more challenging than that of a simple noun. Furthermore, the amount of NEs is not as stable as the vocabulary of a language, but increases fast. The focus of the MultiCoNER shared task (Fetahu et al., 2023b) is to address this challenge in mono- lingual cases as well as in the multilingual case. The main challenges this year are the fine-grained entity taxonomy with over 30 classes and the in- duced noise in the test dataset, whereby the task becomes considerably harder. 2 Related Work The same task was already organized last year at SemEval-2022. Malmasi et al. (2022b) presented SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER), which is a shared task focusing on the identification of complex named entities like media titles, products, and groups in text documents across multiple languages and domains. They also discussed the evaluation metrics used in the task, including the F1 score, and presented baseline results for each language and domain. For each language in the dataset the participants developed models that can identify the boundaries of complex named entities, classify them into predefined entity types, and recognize nested entities and entity modifiers. In general, the top-performing systems for each language and domain used neural network models, such as transformers like BERT (Devlin et al., 2019) and XLM-RoBERTa (Conneau et al., 2020), and ensemble models that combine multiple neural network models and made use of multilingual pre-training or language-specific pre-training to improve the performance of the model. The baseline result for English in MultiCoNER 2022 was a F1 score of 61.36% and the …

Content truncated to 500 words