Will_Go at SemEval-2020 Task 3: An Accurate Model for Predicting the (Graded) Effect of Context in Word Similarity Based on BERT

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 301–306 Barcelona, Spain (Online), December 12, 2020. 301 Will Go at SemEval-2020 Task 3: An Accurate Model for Predicting the (Graded) Effect of Context in Word Similarity based on BERT Wei Bao∗ Southeast University willinseu@gmail.com Hongshu Che∗ Southeast University hsche1222@gmail.com Jiandong Zhang∗ Southeast University zjdx1998@gmail.com Abstract Natural Language Processing (NLP) has been widely used in the semantic analysis in recent years. Our paper mainly discusses a methodology to analyze the effect that context has on human perception of similar words, which is the third task of SemEval 2020. We apply several methods in calculating the distance between two embedding vector generated by Bidirectional Encoder Representation from Transformer (BERT). Our team will go won the 1st place in Finnish language track of subtask1, the second place in English track of subtask1. 1 Introduction Computing the meaning difference between words in the semantic level is a task that has been widely discussed. In the area of natural language processing (NLP) like information retrieval (IR), there are many specific applications using similarity, such as text memorization (Lin and Hovy, 2003), text categorization (Ko et al., 2004), Text Q&A (Mohler and Mihalcea, 2009), etc. The task3 of SemEval-20201 focuses on the influence of context when humans perceive similar words (Armendariz et al., 2020a). As we all know, polysemous words have different meanings in a totally different context, which the current translation system can recognize very well. However, many translation systems can’t exactly predict the subtle variance on the meanings of words, which is also caused by a different context. Task3 has two sub-tasks. In subtask1, we are required to predict the extent of change in scores of similarity between two words in different contexts by human annotators. In subtask2, we only predict the absolute score as is in the subtask1 rather than the difference in scores, and we would only discuss subtask1. Our team uses different algorithms to calculate the distance between two embedding vectors generated by BERT (Devlin et al., 2018) and defines it as the similarity. So we can get the change in similarity by subtraction between two distances. However, this methodology did not get exciting performance in the task evaluation, so we improve this by blending different BERT models, which we will introduce later in Section 3.2. 2 Related Work There are many methods and models to estimate the similarity between long paragraphs. Most of them treat it as a binary classification problem, Hatzivassiloglou et al. (Hatzivassiloglou et al., 1999) compute the linguistic vector of features including primitive features and composite features, then they build criteria by feature vectors to classify paragraphs. As for similarity between short sentences, Foltz et al.(Foltz et al., 1998) suggest a method that provides a characterization of the extent of semantic similarity between two adjacent short sentences by comparing their high-dimensional semantic vectors, which is also a Latent Semantic Analysis (LSA) model. Both LSA and Hyperspace Analogues to Language This work is licensed under a Creative …

Content truncated to 500 words