OCHADAI at SemEval-2022 Task 2: Adversarial Training for Multilingual Idiomaticity Detection

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 217 - 220 July 14-15, 2022 ©2022 Association for Computational Linguistics OCHADAI at SemEval-2022 Task 2: Adversarial Training for Multilingual Idiomaticity Detection Lis Kanashiro Pereira1, Ichiro Kobayashi2 Ochanomizu University kanashiro.pereira@ocha.ac.jp1, koba@is.ocha.ac.jp2 Abstract We propose a multilingual adversarial training model for determining whether a sentence con- tains an idiomatic expression. Given that a key challenge with this task is the limited size of annotated data, our model relies on pre-trained contextual representations from different multi- lingual state-of-the-art transformer-based lan- guage models (i.e., multilingual BERT and XLM-RoBERTa), and on adversarial training, a training method for further enhancing model generalization and robustness. Without rely- ing on any human-crafted features, knowledge bases, or additional datasets other than the tar- get datasets, our model achieved competitive results and ranked 6th place in SubTask A (zero- shot) setting and 15th place in SubTask A (one- shot) setting. 1 Introduction Large-scale pre-trained language models such as BERT (Devlin et al., 2019) have achieved great success in a wide range of natural language pro- cessing (NLP) tasks. However, more recent studies show that even such contextual models have a lim- ited ability to capture idiomaticity (Garcia et al., 2021). Idiomatic expressions denote a group of words that behave as single words to some extent. Their linguistic behavior cannot be inferred from the characteristics of their components, and still pose a challenge to natural language processing (NLP) systems. This paper describes the system developed by the OCHADAI team for SemEval-2022 Task 2 - Multilingual Idiomaticity Detection and Sentence Embedding (Tayyar Madabushi et al., 2022). Given that a key challenge in this task is the limited size of annotated data, we follow best practices from recent work on enhancing model generalization and robustness and propose a model ensemble that leverages multilingual pre-trained representations and adversarial training. Our model ranked 6th Setting Language Train Dev Eval Test zero-shot English 3,327 - - - Portuguese 1,164 - - - Galician 0 - - - one-shot English 87 466 483 916 Portuguese 53 273 279 713 Galician 0 0 0 713 Table 1: Summary of the SemEval 2022 Task 2 Subtask A dataset. Note that the dev, eval and test sets are used in both settings. on SubTask A (zero-shot), and 15th on SubTask A (one-shot). 2 Task Description SemEval-2021 Task 2 SubTask A consists of a bi- nary classification task that requires classifying sen- tences with a target multiword expression (MWE) into either "Idiomatic" or "Literal" across English, Portuguese and Galician (Tayyar Madabushi et al., 2021). Further, it is subdivided into two settings to better test models’ ability to generalize: zero-shot and one-shot. In the zero-shot" setting, multiword expressions (potentially idiomatic phrases), in the training set are completely disjoint from those in the test and development sets. In the "one-shot" setting, one positive and one negative training ex- amples are included for each MWE in the test and development sets. Note that the actual examples in the training data are different …

Content truncated to 500 words