LLM-RM at SemEval-2023 Task 2: Multilingual Complex NER Using XLM-RoBERTa

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 453–456 July 13-14, 2023 ©2023 Association for Computational Linguistics LLM-RM at SemEval-2023 Task 2: Multilingual Complex NER using XLM-RoBERTa Rahul Mehta IIIT Hyderabad,India rahul.mehta@research.iiit.ac.in Vasudeva Varma IIIT Hyderabad,India vv@iiit.ac.in Abstract Named Entity Recognition(NER) is a task of recognizing entities at a token level in a sen- tence. This paper focuses on solving NER tasks in a multilingual setting for complex named en- tities.Our team, LLM-RM participated in the recently organized SemEval 2023 task, Task 2: MultiCoNER II,Multilingual Complex Named Entity Recognition. We approach the problem by leveraging cross-lingual representation pro- vided by fine-tuning XLM-Roberta base model on datasets of all of the 12 languages pro- vided - Bangla, Chinese, English, Farsi, French, German, Hindi, Italian, Portuguese, Spanish, Swedish and Ukrainian. 1 Introduction Named Entity Recognition(NER) is the task of rec- ognizing entities (e.g., person, location, organiza- tion) in a piece of text. Most of the NER datasets like CoNLL2003 (Erik F. Tjong and Meulder, 2003) are focussed on high- resource languages like English and are specific to a given domain like news. The SemEval-2023 Task 2, Multilingual Complex Named Entity Recogni- tion (MultiCoNER II) (Fetahu et al., 2023b) con- tains NER datasets curated for 12 languages which are low resource and from other domains as well. The task also provided a large number of 30 classes and simulated errors being added to the test set to make it more challenging. It contains the following languages: English, Spanish, Dutch, Russian, Turk- ish, Portugues, Korean, Farsi, German, Chinese, Hindi, and Bangla. With the advent of Deep Learning based mod- els, Transformer models like BERT, and bidirec- tional LSTM based ELMo became state of the art. BERT’s multilingual counterpart, mBERT became state of the art in multilingual NER tasks. Another model XLM-RoBERTa(XLM-R) (Alexis Conneau and Stoyanov, 2020) has shown to outperform mBERT on low resource languages on the NER tasks. XLM-R is suitable for our task as it is pre- trained for more than 100 languages including the 4 languages which we worked upon. Therefore, we have used XLM-RoBERTa-base transformer model and fine-tuned it on each of the given 12 languages. 1.1 The Challenge We participated in the shared task named Multi- CoNER II, which is part of Semeval 2023. The task consists of building a named entity recognition sys- tem in 12 languages. There are 2 tracks to the tasks, one is the monolingual track for each language and the other is a multilingual track consisting of all languages. The condition of the competition was that one can only use the prediction of the mono- lingual model in the monolingual track and cannot use it for the multilingual track The task is a successor to the 2022 challenge of MultiCoNER shared task, (Malmasi et al., 2022b) where the key challenges were the following 1) syntactically complex entities like movies titles e.g Shawshank Redemption 2) these complex entities having low context and 3) long-tail entity distribu- tions. These challenges of NER …

Content truncated to 500 words