hub at SemEval-2021 Task 2: Word Meaning Similarity Prediction Model Based on RoBERTa and Word Frequency

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 719–723 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 719 hub at SemEval-2021 Task 2: Word Meaning Similarity Prediction Model Based on RoBERTa and Word Frequency Bo Huang, Yang Bai, Xiaobing Zhou* School of Information Science and Engineering Yunnan University, Yunnan, P.R. China *Corresponding author:zhouxb@ynu.edu.com Abstract This paper introduces the system description of the hub team, which explains the related work and experimental results of our team’s participation in SemEval 2021 Task 2: Mul- tilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC). The data of this shared task is mainly some cross-language or multi-language sentence pair corpus. The lan- guages covered in the corpus include English, Chinese, French, Russian, and Arabic. The task goal is to judge whether the same words in these sentence pairs have the same meaning in the sentence. This can be seen as a task of binary classification of sentence pairs. What we need to do is to use our method to deter- mine as accurately as possible the meaning of the words in a sentence pair are the same or dif- ferent. The model used by our team is mainly composed of RoBERTa and Tf-Idf algorithms. The result evaluation index of task submission is the F1 score. We only participated in the En- glish language task. The final score of the test set prediction results submitted by our team was 84.60. 1 Introduction and Background With the continuous development of science and technology, we are now in an era of massive data. We cannot use manual methods in the processing and retrieval of text data. Especially in the work of comparing and calculating the semantic differ- ence at the word level in the text. In this type of work, automatic processing of text data with ma- chines has become a new choice. The research on the detection method (Resnik, 1995; Miller and Charles, 1991) and evaluation method (S´anchez et al., 2012) of semantic similarity has become a subject of wide discussion. Specific application scenarios have been produced in some fields of natural language processing and information re- trieval. Such as sentiment analysis (Araque et al., Figure 1: A word cloud diagram of the training set text data provided by the task organizer team. The result shown in the figure is the data after removing the stop words. 2019), medical disease similarity query (Mathur and Dinakarpandian, 2012), text question and an- swer(Mohler and Mihalcea, 2009) etc. Similar to humans’ strategies for detecting the meaning of words in different sentences, machines and algorithms also need to predict the results based on the context. Therefore, the method of gen- erating vectors based on each word is not suitable for such tasks. For example Word2Vec (Mikolov et al., 2013). Based on the characteristics of text se- rialization, extracting contextual information in the text as the input of the model will provide the model with richer and more accurate information. For ex- ample, in dealing with the problem …

Content truncated to 500 words