AlexU-AUX-BERT at SemEval-2020 Task 3: Improving BERT Contextual Similarity Using Multiple Auxiliary Contexts

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 270–274 Barcelona, Spain (Online), December 12, 2020. 270 AlexU-AUX-BERT at SemEval-2020 Task 3: Improving BERT Contextual Similarity Using Multiple Auxiliary Contexts Somaia Mahmoud Alexandria University Egypt somaia.0246@gmail.com Marwan Torki Alexandria University Egypt mtorki@alexu.edu.eg Abstract This paper describes the system we built for SemEval-2020 task 3. That is predicting the scores of similarity for a pair of words within two different contexts. Our system is based on both BERT embeddings and WordNet. We simply use cosine similarity to find the closest synset of the target words. Our results show that using this simple approach greatly improves the system behavior. Our model is ranked 3rd in subtask-2 for SemEval-2020 task 3. 1 Introduction A word meaning can be affected by its context. For polysemous words, the change in meaning is usually clear and evident. But even for words that are not necessarily considered polysemous, there can be subtle changes in their meaning. In SemEval-2020 Task 3 (Armendariz et al., 2020a), one goal is to predict how similar two words are in a given context, specifically given two different shared contexts for each pair of target words. Identifying the meaning of words in context is known as Word Sense Disambiguation (WSD). It is a core task of Natural Language Processing (NLP) and has many potential applications. In (Navigli, 2009), the authors presented the motivations for solving the ambiguity of words. Example 1 Context 1 Small arms include handguns, rifles, machine guns, etc. Context 2 He stretched his arms and rifled through the drawer Example 2 Context 1 He proposed a simple solution to solve the problem. Context 2 He promised him to solve his school debt if he found the right ratios of the chemical solution. Figure 1: Input example: There are two target words (boldfaced). These target words are presented in two different contexts. We have participated in Task-3 Subtask-2. In this task, given two target words in a shared context, we want to score how similar they are. We then put the same two words in a different context and re-score their similarity. The input to our system is a pair of words within two different contexts. The output would be the two similarity scores, one for each context. Figure 1 shows the input examples. We consider the words ‘rifle’ and ‘arm’ as our target words. In the first context, the words refer to the same meaning, a weapon. In the second context, they refer to different senses, (‘arm’ as a body part, and ‘rifle’ meaning ‘to search’). Similarly, in the first context, the words ‘solution’ and ‘solve’ have the same sense, solving a problem. In the second context, ‘solution’ means a chemical mixture and ‘solve’ means to clear a debt. This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http://creativecommons.org/licenses/by/4.0/ 271 In this paper, we tackle the problem of multiple senses for target words. Inspired by (Levine et al., 2019), we compose multiple embeddings for …

Content truncated to 500 words