Wolfies at SemEval-2022 Task 8: Feature extraction pipeline with transformers for Multi-lingual news article similarity

Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 135 - 144 July 14-15, 2022 ©2022 Association for Computational Linguistics Hitachi at SemEval-2022 Task 2: On the Effectiveness of Span-based Classification Approaches for Multilingual Idiomaticity Detection Atsuki Yamaguchi, Gaku Morio, Hiroaki Ozaki and Yasuhiro Sogawa Research and Development Group, Hitachi, Ltd. Kokubunji, Tokyo, Japan {atsuki.yamaguchi.xn, gaku.morio.vn, hiroaki.ozaki.yu, yasuhiro.sogawa.tp}@hitachi.com Abstract In this paper, we describe our system for SemEval-2022 Task 2: Multilingual Idiomatic- ity Detection and Sentence Embedding. The task aims at detecting idiomaticity in an input sequence (Subtask A) and modeling represen- tation of sentences that contain potential id- iomatic multiword expressions (MWEs) (Sub- task B) in three languages. We focus on the zero-shot setting of Subtask A and propose two span-based idiomaticity classification methods: MWE span-based classification and idiomatic MWE span prediction-based classification. We use several cross-lingual pre-trained language models (InfoXLM, XLM-R, and others) as our backbone network. Our best-performing sys- tem, fine-tuned with the span-based idiomatic- ity classification, ranked fifth in the zero-shot setting of Subtask A and exhibited a macro F1 score of 0.7466. 1 Introduction SemEval-2022 Task 2 (Tayyar Madabushi et al., 2022) involves detecting idiomaticity in a given sentence (Subtask A) and learning effective repre- sentations of sentences that may contain idiomatic multiword expressions (MWEs) (Subtask B) in three languages: English, Portuguese, and Galician. Processing idiomaticity in a sequence correctly is an essential task in natural language processing (NLP), as idiomatic expressions are a key compo- nent of natural languages. Its high performance will contribute to various downstream tasks, such as sentiment analysis, information retrieval, and machine translation (Hashempour and Villavicen- cio, 2020; Tayyar Madabushi et al., 2021). In this work, we propose two different ap- proaches for multilingual idiomaticity detection (Subtask A) that take advantage of MWE span- based features. We use several cross-lingual pre- trained language models (InfoXLM (Chi et al., 2021a), XLM-R (Conneau et al., 2020), and others) and exploit their MWE span representations for classification, instead of using a special classifica- tion token ([CLS]), which typically corresponds to the first input token. Our concept is that these models should be able to focus more on the id- iomaticity of an MWE in a given sequence by using its span representation rather than using the [CLS] token for classification, potentially resulting in a better detection performance. Our main findings in the shared task are in three- fold. 1. The span-based idiomaticity classification method is highly effective compared to the standard [CLS]-based sequence classifica- tion approach adopted in various BERT (De- vlin et al., 2019)-like models (Liu et al., 2019; Lan et al., 2020; Clark et al., 2020). 2. Detecting idiomaticity in Galician with no training data available is challenging even with state-of-the-art cross-lingual pre-trained language models. 3. Utilizing adjacent contexts with a target sen- tence is not always the best option for id- iomaticity detection, even though it improves the baseline performance. Consequently, our best-performing system, using the span-based classification, ranked fifth among 20 systems in the zero-shot setting of …

Content truncated to 500 words