UA-KO at SemEval-2022 Task 11: Data Augmentation and Ensembles for Korean Named Entity Recognition

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1608 - 1612 July 14-15, 2022 ©2022 Association for Computational Linguistics UA-KO at SemEval-2022 Task 11: Data Augmentation and Ensembles for Korean Named Entity Recognition Hyunju Song, Steven Bethard University of Arizona {hyunjusong,bethard}@email.arizona.edu Abstract This paper presents the approaches and systems of the UA-KO team for the Korean portion of SemEval-2022 Task 11 on Multilingual Com- plex Named Entity Recognition. We fine-tuned Korean and multilingual BERT and RoBERTA models, conducted experiments on data aug- mentation, ensembles, and task-adaptive pre- training. Our final system ranked 8th out of 17 teams with an F1 score of 0.6749 F1. 1 Introduction Named Entity Recognition (NER) is the task of recognizing and classifying named entities in un- structured text. NER is a critical component for NLP tasks such as question answering and relation extraction. Recent advances in neural NER have allowed state-of-the-art systems to perform well in recognizing persons, locations, and organizations in a variety of benchmark datasets. However, these systems still struggle to recognize complex named entities such as titles of creative works. SemEval Task 11 (Malmasi et al., 2022b) asks participants to build systems to identify both classic and complex named entities in multiple languages. In this paper, we describe the UA-KO team’s approach to deal with the challenge of recogniz- ing complex named entities in Korean. We used both monolingual and multilingual transformer- based models. To improve the performance of the models, we conducted experiments on data augmentation, ensembles, and task-adaptive pre- training. Our final performance on the test set ranked 8th out of 17 teams with an F1 score of 0.675. Code to replicate our experiments is avail- able at https://github.com/hyunssong/semeval2022- task11. 2 Related Work NER in real-world settings is challenging. Identify- ing entities in short texts, such as search queries on the web, is not easy due to the lack of context. Iden- tifying nontraditional named entities such as titles of creative works (movies, books, or TV shows) is challenging because similar phrases appear as non- named entities and the characteristics of creative work titles change over time (Ashwini and Choi, 2014). Approaches to such real-world named entity types include integrating English Wikipedia as a gazetteer within neural NER models (Meng et al., 2021), though simply relying on the robustness of large pre-trained models like XLM-R is more successful (Ushio and Camacho-Collados, 2021). SemEval-2022 Task 11 asks participants to build systems that can perform well in real-world set- tings, providing a dataset with complex named en- tities in short sentences. The task is divided into three tracks: multi-lingual, monolingual, and code- mixed. We participated in the Korean portion of the monolingual track. Korean is an agglutinative language, where each word (eojeol) is formed by combining a root mor- pheme with a bound morpheme (josa) or postpo- sition (eomi) (Choi et al., 2017). More than 60 different forms can thus be created from each root morpheme. A model that is not aware of this mor- phological richness may end …

Content truncated to 500 words