UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1511 - 1517 July 14-15, 2022 ©2022 Association for Computational Linguistics UM6P-CS at SemEval-2022 Task 11: Enhancing Multilingual and Code-Mixed Complex Named Entity Recognition via Pseudo Labels using Multilingual Transformer Abdellah El Mekki1 Abdelkader El Mahdaouy1 Mohammed Akallouch1,2 Ismail Berrada1 Ahmed Khoumsi 3 1School of Computer Sciences, Mohammed VI Polytechnic University, Morocco 2Faculty of Sciences Dhar EL Mahraz, Sidi Mohamed Ben Abdellah University, Morocco 3Dept. Electrical & Computer Engineering, University of Sherbrooke, Canada {firstname.lastname}@um6p.ma Abstract Building real-world complex Named Entity Recognition (NER) systems is a challenging task. This is due to the complexity and ambi- guity of named entities that appear in various contexts such as short input sentences, emerg- ing entities, and complex entities. Besides, real-world queries are mostly malformed, as they can be code-mixed or multilingual, among other scenarios. In this paper, we introduce our submitted system to the Multilingual Com- plex Named Entity Recognition (MultiCoNER) shared task. We approach the complex NER for multilingual and code-mixed queries, by re- lying on the contextualized representation pro- vided by the multilingual Transformer XLM- RoBERTa. In addition to the CRF-based to- ken classification layer, we incorporate a span classification loss to recognize named entities spans. Furthermore, we use a self-training mechanism to generate weakly-annotated data from a large unlabeled dataset. Our proposed system is ranked 6th and 8th in the multilingual and code-mixed MultiCoNER’s tracks respec- tively. 1 Introduction Recent named entity recognition (NER) models have achieved great performance for many lan- guages and using various benchmark datasets such as CoNLL2003 and OntoNotes 5.0 (Devlin et al., 2019). However, it is unclear whether or not these systems can handle ambiguous and complex enti- ties, especially in the case of short and low-context settings (Augenstein et al., 2017). It is also un- clear weather these systems can be deployed in real-world scenarios where the input data can be in different languages or code-mixed (Luken et al., 2018; Hanselowski et al., 2018). In fact, to illus- trate these issues, if consider the example of com- plex named entities such as the titles of creative works (movies, songs, books ...), they are hard to be recognized by simple NER systems. This is due to their syntactic ambiguity and the form they can take from one context to another. For instance, they can be as an imperative clause ("Dial M for Murder") or a proposition ("On the beach") which refers to the name of a movie. Thus, it is important to check the performance of NER systems in these scenarios. The complexity of named entities can be due to three main reasons: 1. Complex entities: These entities can be repre- sented as complex infinitives (To Kill a Mock- ingbird) or full clauses (Mr.Smith Goes to Washington). Additionally, they can be rep- resented as noun phrases or gerunds. State- of-the-art systems (Aguilar et al., 2017) have shown that it is hard to recognize such entities. 2. Ambiguous entities and contexts: These types of entities …

Content truncated to 500 words