IXA at SemEval-2023 Task 2: Baseline Xlm-Roberta-base Approach

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 379–381 July 13-14, 2023 ©2023 Association for Computational Linguistics IXA at SemEval-2023 Task 2: Baseline Xlm-Roberta-base Approach Edgar Andrés Santamaría University of the Basque Country (UPV/EHU) HiTZ Center, IXA Research group Datu(a) IA edgar.andres.santamaria@gmail.com Abstract IXA proposes a Sequence labeling fine-tune approach, which consists of a lightweight baseline (10e), the system takes advantage of transfer learning from pre-trained Named Entity Recognition and cross-lingual knowl- edge from the LM checkpoint. This technique obtains a drastic reduction in the effective training costs that works as a perfect baseline, future improvements in the baseline approach could fit: 1) Domain adequation, 2) Data augmentation, and 3) Intermediate task learning. Practical Information The system is composed of two main stages, first of all, fine-tuning which consists in learning IOB sequences via sequence classification task, and then the inference phase when unannotated examples are given to guess the correct IOB sequence. We learn the correct way to label sequences using real examples from the training set and back- propagate the labeling errors over the development set compared to the intended gold standard. At this point, we are taking advantage of already pre-trained checkpoints over MLM and Sequence Classification tasks. Figure 1: Trainning Schema. Finally, we infer the proper IOB sequences for raw examples, those inferences are led by Lan- guage Model expertise on previous sequence clas- sification plus the fine-tuning on MulticoNER this property is called transfer learning, additionally the Masked Language Modelling allows the unseen token representation into the already known word space via substrings in this case, this property is called cross linguality. Figure 2: Inference Schema. 1 Introduction The tag set of MultiCoNER is a fine-grained taxonomy 3 with 36 tags divided into 6 main groups. Further relevant information related to SemEval shared tasks is provided in guidelines (Fetahu et al., 2023). In addition to multilingualism, there were two key challenges in this year’s task: (1) a fine-grained entity taxonomy which mainly penalizes the preci- sion of systems as the ability to guess the correct tag, and (2) simulated errors added to the test set to make the task more realistic and difficult which mainly penalizes the recall of systems as the ability to guess the correct chunk of text to tag. 2 Related Work SemEval-2023 Task 2 follows the multilingual NER task started in 2022 (Malmasi et al., 2022b). critical challenges in the 2022 datasets were dealing with complex entities with limited context (Malmasi et al., 2022a). This is commonly understood as a low-resourced specific domain problem e.g "Medical Entity Recognition", main 379 issues come from the fact that only a few of the total examples are annotated and the provided contexts show specific lexical usually unknown for language models e.g "in particular he worked on the laser photocoagulation of threshold retinopathy of prematurity". The challenges of Sequence Classification for recognizing complex entities in low-context conditions was recently outlined by Meng et al. (2021). Other work has extended the content …

Content truncated to 500 words