UC3M-PUCPR at SemEval-2022 Task 11: An Ensemble Method of Transformer-based Models for Complex Named Entity Recognition

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1448 - 1456 July 14-15, 2022 ©2022 Association for Computational Linguistics UC3M-PUCPR at SemEval-2022 Task 11: An Ensemble Method of Transformer-based Models for Complex Named Entity Recognition Elisa Terumi Rubel Schneider1, Renzo M. Rivera-Zavala2, Paloma Martinez2, Claudia Moro1 and Emerson Cabrera Paraiso1 1Graduate Program on Informatics, Pontifícia Universidade Católica do Paraná, Brazil paraiso@ppgia.pucpr.br 2Computer Science and Engineering Department, Universidad Carlos III de Madrid, Spain pmf@inf.uc3m.es Abstract This study introduces the system submitted to the SemEval 2022 Task 11: MultiCoNER (Mul- tilingual Complex Named Entity Recognition) by the UC3M-PUCPR team. We proposed an ensemble of transformer-based models for en- tity recognition in cross-domain texts. Our deep learning method benefits from the trans- former architecture, which adopts the attention mechanism to handle the long-range dependen- cies of the input text. Also, the ensemble ap- proach for named entity recognition (NER) im- proved the results over baselines based on indi- vidual models on two of the three tracks we par- ticipated in. The ensemble model for the code- mixed task achieves an overall performance of 76.36% F1-score, a 2.85 percentage point in- crease upon our individually best model for this task, XLM-RoBERTa-large (73.51%), outper- forming the baseline provided for the shared task by 18.26 points. Our preliminary results suggest that contextualized language models ensembles can, even if modestly, improve the results in extracting information from unstruc- tured data. 1 Introduction Named Entity Recognition (NER) is a Natural Lan- guage Processing (NLP) technique to extract rel- evant information from unstructured natural lan- guage data, identifying and categorizing entities in texts and thereby supporting other NLP tasks. However, processing complex and ambiguous entities is a challenging task that has not received sufficient attention from the research community (Meng et al., 2021). These complex entities can be formed by any linguistic constituent (long noun phrases and sometimes complete sentences), as ti- tles of creative works such as books and movies, different from traditional entities like person names and locations. As the creative work titles can be se- mantically ambiguous, the challenge is recognizing entities based mainly on their context. The SemEval 2022 Task 11: MultiCoNER (Multilingual Complex Named Entity Recognition) (Malmasi et al., 2022b) targets the recognition of lowercase complex entities in multilingual and multi-domain texts, encouraging researchers to de- velop new approaches to extract diversified entity types. The challenge involves the use of human lan- guage modeling in the NER task, on cross-domain texts. The semantic structure has six types of enti- ties: person, location, group, corporation, product and creative work. The dataset was provided for 11 languages, and, in addition, this task also pro- vided multi-language and code-mixed 1 features, encouraging the development of more generic and adaptive systems. As the contextualized pre-trained language mod- els based on the transformer architecture have reached the state-of-the-art in several NLP tasks (Vaswani et al., 2017), in our method, we explore transformer-based models and combined the results into an ensemble, which can make better predic- tions …

Content truncated to 500 words