SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor!

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 90–97 Barcelona, Spain (Online), December 12, 2020. 90 JCT at SemEval-2020 Task 1: Combined Semantic Vector Spaces Models for Unsupervised Lexical Semantic Change Detection Efrat Amar, Chaya Liebeskind Department of Computer Science, Jerusalem College of Technology, Lev Academic Center 21 Havaad Haleumi St., P.O.B. 16031 9116001 Jerusalem, Israel efrati.amar@gmail.com, liebchaya@gmail.com Abstract In this paper, we present our contribution in SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection, where we systematically combine existing models for unsupervised capturing of lexical semantic change across time in text corpora of German, English, Latin and Swedish. In particular, we analyze the score distribution of existing models. Then we define a general classification threshold, adjust it independently to each of the models and measure the models’ score certainty. Finally, using both the threshold and score certainty, we aggregate the models for the two sub-tasks: binary classification and ranking. 1 Introduction Over the last decade, research on detection of lexical semantic change has increased. Many studies were performed on various languages, corpora and periods. Two years ago, two literature surveys on computational approaches to Lexical Semantic Change (LSC) (Kutuzov et al., 2018; Tahmasebi et al., 2018) were published. Last year Schlechtweg et al. (2019) first systematically compared a broad variety of LSC detection models on two data sets of different periods and domains and Shoemark et al. (2019) proposed a new evaluation framework for semantic change detection using word embeddings. To facilitate the comparison of different systems, SemEval-2020 Task 1 (Schlechtweg et al., 2020) introduced a simple evaluation framework for unsupervised lexical semantic change detection in text corpora of German, English, Latin and Swedish. The task relies on the comparison of two time periods for each language. We participated in two sub-tasks: a classification task, where we decide which words lost or gained senses between the periods, and a ranking task, where we rank a set of target words according to their degree of lexical semantic change between the periods. Given the large number of models that have already been explored, we built a system which systemati- cally combines existing models in the unsupervised setting of the LSC detection task. Since no tuning data is available, we minimized the parameters number of our system. This paper is organized as follows: First, in Section 2, we describe the existing LSC detection models that we have combined. Then, in Section 3, we analyze the score distribution of the models in order to learn the general behaviour of words in our corpora. We aim to estimate the amount of words that changed their meaning between the periods. Next, we define a general classification threshold percentile (CT) parameter and adapt it to each model separately. We use the CT parameter to measure the models’ score certainty too, and filtered models with certainty low than a minimal required decision certainty. The minimal required decision certainty (MCR) is also a parameter of our system. Finally, we present our aggregation methods for the two …

Content truncated to 500 words