Ferryman at SemEval-2020 Task 3: Bert with TFIDF-Weighting for Predicting the Effect of Context in Word Similarity

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 281–285 Barcelona, Spain (Online), December 12, 2020. 281 Ferryman at SemEval-2020 Task 3: Bert with TFIDF-Weighting for Predicting the Effect of Context in Word Similarity Weilong Chen, Xin Yuan, Sai Zhang, Jiehui Wu, Yanru Zhang, and Yan Wang∗ University of Electronic Science and Technology of China chenweilong1995,coma uestc,dwight12,jiehuiwu@std.uestc.edu.cn yanruzhang,yanbo1990@uestc.edu.cn Abstract Word similarity is widely used in machine learning applications like searching engine and recom- mendation. Measuring the changing meaning of the same word between two different sentences is not only a way to handle complex features in word usage (such as sentence syntax and semantics), but also an important method for different word polysemy modeling. In this paper, we present the methodology proposed by team Ferryman. Our system is based on the Bidirectional Encoder Representations from Transformers (BERT) model combined with term frequency-inverse docu- ment frequency (TF-IDF), applying the method on the provided datasets called CoSimLex, which covers four different languages including English, Croatian, Slovene, and Finnish. Our team Ferryman wins the the first position for English task and the second position for Finnish in the subtask 1. 1 Introduction Word similarity quantitatively measures semantic similarity of two words by comparing word vectors or word embedding, multi-dimensional meaning representations of two words. Word embedding models can be trained and used to predict the value of word similarity, which are comparable to methods based on human judgments. However, most of these established methods are context-independent. Context plays a significant role for people to understand the meaning of words from different perspectives. The meaning of an ambiguous word can be inferred by a certain context, which is known as contextual selection of senses. In a single sense, context can modify the meaning of a word by highlighting certain semantic traits and backgrounding others, which is called the contextual modulation of meaning. To fully utilize context and avoid treating words in isolation, a model that can reflect similarity judgement of words in context is required, which takes word sense, discrete effects and other factors of context into consideration(Armendariz et al., 2020a). In this paper, we primarily focus on quantifying the similarity of two words in different contexts. First, literature surveys have been conducted to obtain the similarity scores of two words in two different contexts. Then, the uncentered Pearson correlation coefficient between the scores output by our model and the scores output by the human annotators are calculated. According to the Pearson correlation coefficient, we can evaluate the predicted performance of our context-dependent embeddings model in human perception of similarity. We have pre-processed the data by replacing abbreviation and used TF-IDF(Das, 2007) to enrich the attributes of the pair of words to be evaluated in the mask layer of the bert. Then we fine-tune the BERT model(Devlin et al., 2018) with multi-dropout for predicting the absolute similarity rating for each pair of words in each context. Among all teams participating in CoSimLex, our models ranks the 1st place out of 14 on subtask …

Content truncated to 500 words