JUSTMasters at SemEval-2020 Task 3: Multilingual Deep Learning Model to Predict the Effect of Context in Word Similarity

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 292–300 Barcelona, Spain (Online), December 12, 2020. 292 JUSTMasters at SemEval-2020 Task 3: Multilingual Deep Learning Model to Predict the Effect of Context in Word Similarity Nour Al-Khdour Dept. of Computer Science Jordan University of Science and Technology-Irbid, Jordan naalkhdour17@cit.just.edu.jo Mutaz Bni Younes Dept. of Computer Science Jordan University of Science and Technology-Irbid, Jordan mmbniyounes18@cit.just.edu.jo Malak Abdullah Dept. of Computer Science Jordan University of Science and Technology-Irbid, Jordan mabdullah@just.edu.jo Mohammad AL-Smadi Dept. of Computer Science Jordan University of Science and Technology-Irbid, Jordan maalsmadi9@just.edu.jo Abstract There is a growing research interest in studying word similarity. Without a doubt, two similar words in a context may be considered different in another context. Therefore, this paper investi- gates the effect of the context in word similarity. The SemEval-2020 workshop has provided a shared task (Task 3: Predicting the (Graded) Effect of Context in Word Similarity). In this task, the organizers provided unlabeled datasets for four languages, English, Croatian, Finnish, and Slovenian. Our team, JUSTMasters, has participated in this competition in the two subtasks: A and B. Our approach has used a weighted average ensembling method for different pre-trained embeddings techniques for each of the four languages. Our proposed model outperformed the baseline models in both subtasks and achieved the best result for subtask 2 in English and Finnish, with a score of 0.725 and 0.68, respectively. We have been ranked the sixth in subtask 1, with English, Croatian, Finnish, and Slovenian with scores as follows: 0.738, 0.44, 0.546, 0.512. 1 Introduction Identifying and measuring words similarity is considered the base step for spotting the similarity between sentences or documents. Word similarity is classified into three types: the first type is lexical similarity (Gomaa et al., 2013), which depends on the symmetry of the character sequence; the second one is semantic similarity (Miller and Charles, 1991), which depends on the literal meaning of the words; and the third type is pragmatics (Cherpas, 1992), which focuses on the meanings and the effects of words according to the context. Task 3 in SemEval 2020 (Armendariz et al., 2020a) is about finding the context effect on word meanings by giving two similar words in a different context. Task 3 has two unsupervised subtasks, the first subtask asked to predict the degree and direction of change in the human annotator’s scores of similarity between two words when presented in two different contexts. The second subtask asked to predict the human score of similarity for a pair of words within the same context. In addition, the task provided data for four different languages: English, Croatian, Finnish, and Slovenian. In this paper, we have experimented with different embeddings techniques in each of the four languages to determine the best combination for each subtask. Also, we propose a weighted ensembling for state- of-the-art embedding techniques to achieve a significant sense of representation of the words. This has been accomplished for the four languages to gain the highest result in both subtasks …

Content truncated to 500 words