NamedEntityRangers at SemEval-2022 Task 11: Transformer-based Approaches for Multilingual Complex Named Entity Recognition

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1570 - 1575 July 14-15, 2022 ©2022 Association for Computational Linguistics NamedEntityRangers at SemEval-2022 Task 11: Transformer-based Approaches for Multilingual Complex Named Entity Recognition Amina Miftahova Innopolis University, Russia noteisenheim@gmail.com Alexander Pugachev HSE University, Russia avpugachev@edu.hse.ru Artem Skiba Ershov Institute of Informatics Systems, Russia efildream@gmail.com Ekaterina Artemova HSE University Huawei Noah’s Ark lab, Russia elartemova@hse.ru Tatiana Batura Moscow State University Novosibirsk State University, Russia t.batura@g.nsu.ru Pavel Braslavski HSE University Ural Federal University, Russia pbras@yandex.ru Vladimir Ivanov Innopolis University Kazan Federal University, Russia v.ivanov@innopolis.ru Abstract This paper presents the two submissions of NamedEntityRangers Team to the Multi- CoNER Shared Task, hosted at SemEval-2022. We evaluate two state-of-the-art approaches, of which both utilize pre-trained multi-lingual lan- guage models differently. The first approach follows the token classification schema, in which each token is assigned with a tag. The second approach follows a recent template- free paradigm (Ma et al., 2021), in which an encoder-decoder model translates the input se- quence of words to a special output, encoding named entities with predefined labels. We uti- lize RemBERT and mT5 as backbone models for these two approaches, respectively. Our re- sults show that the oldie but goodie token classi- fication outperforms the template-free method by a wide margin. Our code is available at: https://github.com/Abiks/MultiCoNER. 1 Introduction This paper describes two submissions to the Multi- lingual Complex Named Entity Recognition (Mul- tiCoNER) Shared Task, held by SemEval-2022 (Malmasi et al., 2022b). This shared task aims at recognizing named entities with the ambitious end goal of building systems that support up to 11 languages. Multilingual setups are complicated when the dataset mixes languages from different groups. This way, the MultiCoNER dataset com- prises 11 languages from different families, and multiple scripts (Malmasi et al., 2022a). This setup hayj s become increasingly popular recently since it allows to test for transfer learning across lan- guages within a single pre-trained model. The dataset, proposed in the shared task, has several unique features previously neglected in NER eval- uation: syntactically complex entities, ambiguous entities, divers, and low-frequency (aka long-tail) entities. This makes the shared task setup closer to real-life settings, where datasets are way noisier and nonuniformly distributed. Our solution consists of two state-of-the-art ap- proaches adopted to the task. First, we use a main- stream NER technique, token classification. Under this approach, the model is trained to assign a la- bel to each input token. The second approach falls into the group of prompt-based techniques, at the core of which are the capabilities of auto-regressive language models to memorize and reproduce input texts. In this case, we train an encoder-decoder model to replace entities in the input sentence with predefined labels. We utilize RemBERT (Chung et al., 2021) and mT5 (Xue et al., 2021) as back- bone models for these two approaches, respectively. These two models provide state-of-the-art results for the common test-beds of cross-lingual experi- ments (Hu et al., 2020). In other words, we explore the following …

Content truncated to 500 words