Ertim at SemEval-2023 Task 2: Fine-tuning of Transformer Language Models and External Knowledge Leveraging for NER in Farsi, English, French and Chinese

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2211–2215 July 13-14, 2023 ©2023 Association for Computational Linguistics Ertim at SemEval-2023 Task 2: Fine-tuning of Transformer Language Models and External Knowledge Leveraging for NER in Farsi, English, French and Chinese Kévin Deturck, Pierre Magistry, Bénédicte Diot-Parvaz Ahmad, Ilaine Wang, Damien Nouvel ERTIM Inalco / 2 rue de Lille, 75007 Paris Hugo Lafayette Kairntech / 29 Chemin du Vieux Chêne, 38240 Meylan first.last@{inalco,kairntech}.fr Abstract Transformer language models are now a solid baseline for Named Entity Recognition and can be significantly improved by leveraging com- plementary resources, either by integrating ex- ternal knowledge or by annotating additional data. In a preliminary step, this work presents experiments on fine-tuning transformer mod- els. Then, a set of experiments has been con- ducted with a Wikipedia-based reclassification system. Additionally, we conducted a small annotation campaign on the Farsi language to evaluate the impact of additional data. These two methods with complementary resources showed improvements compared to fine-tuning only. 1 Introduction Entity recognition and linking has now become a standard task for NLP, either as a goal in itself or as a preprocessing step for other goals. As for many other NLP tasks, developing an accurate model for a specific dataset (by language or domain) is still a challenge that heavily relies on resources, either pre-computed in a language model or as an external dataset. As the NLP research lab of Inalco, Ertim was particularly interested in MultiCoNER II because it gave us a good opportunity to: • carry out a novel evaluation of HuggingFace models on the complex NER task proposed in MultiCoNER II, through the diversity of more or less endowed languages (Fetahu et al., 2023a) • experiment the integration of external knowl- edge as a postprocessing step to improve the classification of named entities • conduct experiments on NER in Farsi, which coincided with our work for a project (VITAL, see the Acknowledgements section) We report our official results (Fetahu et al., 2023b) in Table 1. Due to lack of time, some of our experiments could not be submitted as official runs, this paper provides additional information and results about these experiments. Lang. Rank F1 clean F1 noisy F1 overall EN 20/34 61.85 52.78 59.03 FR 9/17 69.73 58.6 66.3 FA 12/14 53.77 - 53.77 ZH 11/22 64.26 44.38 59.45 Table 1: Ertim’s rankings for the 4 tracks undertaken 2 Fine-tuning of HuggingFace Language Models We present experiments based on the fine-tuning of transformer language models for the MultiCoNER II task on specific languages. 2.1 Official runs In this section, we present our runs based on the fine-tuning of HuggingFace language models that have been officially retained for the MultiCoNER II final rankings. All these runs have been done by using the Spacy framework. We used models respectively pre-trained on our target languages and fine-tuned them by using the MultiCoNER II train and dev datasets in the corresponding languages. HuggingFace Model Prec. Rec. F1 roberta-base (Liu et al., 2019) 59.13 …

Content truncated to 500 words