KDDIE at SemEval-2022 Task 11: Using DeBERTa for Named Entity Recognition

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 1531 - 1535 July 14-15, 2022 ©2022 Association for Computational Linguistics KDDIE at SemEval-2022 Task 11: Using DeBERTa for Named Entity Recognition Caleb Martin, Huichen Yang, and William Hsu Computer Science of Kansas State University Manhattan, Kansas 66502 {calebjm288, huichen, bhsu}@ksu.edu Abstract In this work, we introduce our system to the SemEval 2022 Task 11: Multilingual Com- plex Named Entity Recognition (MultiCoNER) competition. Our team (KDDIE) attempted the sub-task of Named Entity Recognition (NER) for the language of English in the challenge and reported our results. For this task, we use trans- fer learning method: fine-tuning the pre-trained language models (PLMs) on the competition dataset. Our two approaches are the BERT- based PLMs and PLMs with additional layer such as Condition Random Field. We report our finding and results in this report. 1 Introduction In today’s world there is an ever-growing supply of unstructured data, a lot of it in the form of free text. Named Entity Recognition (NER) is a process that seeks to gather information from free text, by ex- tracting and labeling the named entities. SemEval 2022 Task 11 is a competition where NER systems are trained and then they compete against each other for the best scores (Malmasi et al. (2022b)). The task is split into 13 tracks: one for each of 11 different languages, a multi-lingual track, and a mixed language track (Malmasi et al. (2022b)). In this paper we will be focusing on track 1, which is monolingual English NER. For this SemEval Task participants were asked to have their system both identify named entities and then label them with one of the six provided labels. The six labels to be used were person, location, group, corporation, product, and creative work (Malmasi et al. (2022b)). Figure 1 shows an example of these words being tagged with their appropriate labels. For this paper our strategy involved trying var- ious transformer models from the HuggingFace library (Wolf et al. (2020)) and training them on the training data provided. We then tried adding a conditional random field layer to some different models to see if that would improve the scores that Figure 1: Example of labeling named entities in a sentence using the SemEval 2022 Task 11 tagging scheme. These two sentences are taken directly from the SemEval 2022 Task 11 training set (Malmasi et al. (2022a)). we received. With all the models we trained we also fine-tuned many different training parameters to obtain the best scores possible. We tried using different number of epochs, different learning rates, different batch sizes, and changing many additional parameters. In the research we have done leading up to this paper we have learned many things. One is that adding a CRF layer to a BERT model (Devlin et al. (2018)) helps its performance, but adding a CRF layer to a DeBERTa model (He et al. (2021)) doesn’t help and actually hurts its performance for NER. We also learned through …

Content truncated to 500 words