TEAM-Atreides at SemEval-2022 Task 11: On leveraging data augmentation and ensemble to recognize complex Named Entities in Bangla

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 169 - 177 July 14-15, 2022 ©2022 Association for Computational Linguistics CardiffNLP-Metaphor at SemEval-2022 Task 2: Targeted Fine-tuning of Transformer-based Language Models for Idiomaticity Detection Joanne Boisson and Luis Espinosa Anke and Jose Camacho Collados School of Computer Science and Informatics Cardiff University BoissonJC@cardiff.ac.uk Abstract This paper describes the experiments ran for SemEval-2022 Task 2, subtask A, zero-shot and one-shot settings for idiomaticity detec- tion. Our main approach is based on fine-tuning transformer-based language models as a base- line to perform binary classification. Our sys- tem, CardiffNLP-Metaphor, ranked 8th and 7th (respectively on zero- and one-shot settings on this task. Our main contribution lies in the extensive evaluation of transformer-based lan- guage models and various configurations, show- ing, among others, the potential of large multi- lingual models over base monolingual models. Moreover, we analyse the impact of various in- put parameters, which offer interesting insights on how language models work in practice. 1 Introduction Idiomatic language identification is an important task for language understanding. Recent language models are surprisingly accurate at distinguishing literal and figurative use of language, but very little work has been done on measuring their ability to generalize across languages. Even for mainstream languages such as English, there is still little un- derstanding on the way language models process idiomatic expressions (IE’s). This SemEval task (Tayyar Madabushi et al., 2022) focuses, in partic- ular, on multi-word expressions (MWE), adding the challenge of representing such expressions in models. The Subtask A of SemEval Task 2 invited partic- ipants to extend the range of existing experiments for multilingual idiomatic language detection. Data were provided in English, Portuguese, and Gali- cian. The task is framed as a binary classification of MWE between an idiomatic and a literal us- age. In the zero-shot setting, no Galician example is provided in the training and development set. The MWE of the development, evaluation and test sets are unseen in the training set. In the one-shot setting, exactly one example of the MWE encoun- tered respectively in the practicing and test phase is added to the training data. Therefore, these set- tings provide a challenging framework in which language models have to learn from few or no ex- amples. As part of the CardiffNLP-Metaphor team, we used a simple strategy similar to the method em- ployed in the original paper releasing the dataset Tayyar Madabushi et al. (2021). In particular, we assessed the performance of monolingual and mul- tilingual language models on the task. To this end, we compared the performance of these models us- ing different input formats and training parame- ters. The best results are obtained with a XLM- RoBERTa large (Lample and Conneau, 2019) with 7 epochs, 8 instances per batch, a maximum se- quence length of 350, the longest three-sentence context, and including target information (i.e., the embedding and the position of the target in the sen- tence). Our submitted model was based on the best performance in the …

Content truncated to 500 words