DS4DH at SemEval-2022 Task 11: Multilingual Named Entity Recognition Using an Ensemble of Transformer-based Language Models

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1543 - 1548 July 14-15, 2022 ©2022 Association for Computational Linguistics DS4DH at SemEval-2022 Task 11: Multilingual Named Entity Recognition Using an Ensemble of Transformer-based Language Models Hossein Rouhizadeh University of Geneva, Switzerland hossein.rouhizadeh@unige.ch Douglas Teodoro University of Geneva, Switzerland douglas.teodoro@unige.ch Abstract In this paper, we describe our proposed method for the SemEval 2022 Task 11: Multilingual Complex Named Entity Recognition (Multi- CoNER). The goal of this task is to locate and classify named entities in unstructured short complex texts in 11 different languages. After training a variety of contextual language mod- els on the NER dataset, we used an ensemble strategy based on a majority vote to finalize our model. We evaluated our proposed approach on the multilingual NER dataset at SemEval- 2022. The ensemble model provided consistent improvements against the individual models on the multilingual track, achieving a macro F1 performance of 65.2%. However, our re- sults were significantly outperformed by the top ranking systems, achieving thus a baseline performance. 1 Introduction Named entity recognition (NER) is the process of identifying pre-defined categories of named enti- ties, such as people, places, organizations, from unstructured text. NER usually serves as an im- portant first component in various natural language processing (NLP) tasks, such as question answer- ing (Mollá et al., 2006), information retrieval (Guo et al., 2009) and machine translation (Babych and Hartley, 2003). Thus, the performance of the NER system can influence the quality of many down- stream NLP applications. Despite the high perfor- mance achieved by the current NER systems, they still face some critical challenges(Augenstein et al., 2017). NER models are typically trained on a well- formed news text containing a variety of entities within a relatively long context. In addition, most of the existing NER datasets usually include a large number of common entities between train set and test set. As a result, the performance of the models drops dramatically in the real world applications as they must deal with unseen entities and noisy texts. Furthermore, previous studies on NER have mostly focused on English and as a result, many other languages specially low-resource ones, such as Turkish, Korean, and Persian, have not been as well studied (Rouhizadeh et al., 2021a,b). In this context, SemEval-2022 proposes the task of Multilingual Complex Named Entity Recognition (MultiCoNER) (Malmasi et al., 2022b), which is concerned with detecting semantically ambiguous and complex entities in short and low-contextual settings for 11 languages (i.e. English, Spanish, Dutch, Russian, Turkish, Korean, Farsi, German, Chinese, Hindi, and Bangla). In this paper, we present a multilingual NER method based on en- semble of deep neural language models. We first trained multiple NER models on the official train- ing dataset and then utilized an ensemble strategy based on a majority of votes from the top-3 best- performing models. Based on the macro-average F1-score of 65.2, achieved by our model, we placed 20th in the multilingual track of the competition. The rest of the paper …

Content truncated to 500 words