LIORI at SemEval-2021 Task 2: Span Prediction and Binary Classification approaches to Word-in-Context Disambiguation

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 780–786 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 780 LIORI at SemEval-2021 Task 2: Span Prediction and Binary Classification approaches to Word-in-Context Disambiguation Adis Davletov RANEPA, Moscow, Russia Lomonosov Moscow State University, Moscow, Russia davletov-aa@ranepa.ru Nikolay Arefyev Lomonosov Moscow State University, Moscow, Russia Samsung Research Center Russia, Moscow, Russia HSE University, Moscow, Russia nick.arefyev@gmail.com Denis Gordeev RANEPA, Moscow, Russia gordeev-di@ranepa.ru Alexey Rey RANEPA, Moscow, Russia rey-ai@ranepa.ru Abstract This paper presents our approaches to SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambigua- tion task. The first approach attempted to reformulate the task as a question answering problem, while the second one framed it as a binary classification problem. Our best system, which is an ensemble of XLM-R based binary classifiers trained with data augmentation, is among the 3 best-performing systems for Russian, French and Arabic in the multilingual subtask. In the post-evaluation period, we experimented with batch normal- ization, subword pooling and target word occurrence aggregation methods, resulting in further performance improvements. 1 Introduction In the Semeval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation task, the participants were asked to classify whether the target word, occurring in two sentences (sen- tence1 and sentence2), is used in the same or in a different meaning. The two sentences could be in the same language or different languages. There were two subtasks: • Multilingual Word-in-Context Disam- biguation, where both sentences were in the same language, either Arabic, Chinese, English, French or Russian • Cross-lingual Word-in-Context Disam- biguation, where the first sentence was in English and the second one was either in Arabic, Chinese, French, or Russian. More detailed information regarding the task is provided by Martelli et al. (2021). We participated in both tracks and experimented with two approaches1. The first approach fine-tunes XLM-R (Conneau et al., 2020a) as a question an- swering system searching in the second sentence for a word with the same meaning as the target word in the first sentence. The second approach fine-tunes XLM-R as a binary classifier, and en- sembles several such classifiers. Also, we used data augmentation to double the number of train- ing examples. This second approach took the 2nd place for the monolingual subtask in Arabic and the 3rd place for the monolingual subtask in French and Russian. In the cross-lingual subtask, the sys- tem ranked 6th for French and Arabic. The same system was applied to all subtasks and languages. During the post-evaluation period, we performed thorough experiments with our system. We com- pared different subword pooling methods, includ- ing mean, max, first pooling and their combina- tions, and found that combinations do not help and mean pooling is overall the best choice. Un- like pooling, instead of a simple concatenation of contextualized embeddings for the target word oc- currences, it is helpful to combine their difference and normalized component-wise product. Finally, we found it beneficial to add a batch normalization 1https://github.com/davletov-aa/mcl-wic 781 layer before feeding those vectors into …

Content truncated to 500 words