Masked Reasoner at SemEval-2020 Task 4: Fine-Tuning RoBERTa for Commonsense Reasoning

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 50–58 Barcelona, Spain (Online), December 12, 2020. 50 DiaSense at SemEval-2020 Task 1: Modeling sense change via pre-trained BERT embeddings Christin Beck University of Konstanz christin.beck@uni-konstanz.de Abstract This paper describes DiaSense, a system developed for Task 1 ‘Unsupervised Lexical Semantic Change Detection’ of SemEval-2020. In DiaSense, contextualized word embeddings are used to model word sense changes. This allows for the calculation of metrics which mimic human intuitions about the semantic relatedness between individual use pairs of a target word for the assessment of lexical semantic change. DiaSense is able to detect lexical semantic change in English, German, Latin and Swedish (accuracy = 0.728). Moreover, DiaSense differentiates between weak and strong change. 1 Introduction Task 1 of SemEval-2020 (Schlechtweg et al., 2020) is concerned with the unsupervised detection of lexical semantic change (LSC) as reflected by word sense changes over time. More broadly, LSC refers to changes in meaning of a lexical item. A meaning change is manifested in the gain or loss of a particular meaning of a word which indicates an increase or decrease in polysemy (Traugott and Dasher, 2001; Traugott, 2006). A well-known example for LSC which is cited in Tahmasebi et al. (2018) is the historical evolution of the English word hound which changed from being the general word for ‘dog’ to referring to only a specific kind of ‘dog’ (‘narrowing’, cf. Traugott (2006)). Meanwhile, dog changed from describing a specific type of ‘dog’ to becoming the general term for ‘dog’ (‘broadening’, cf. Traugott (2006)). Moreover, senses can become obsolete and be lost overall, while cultural changes may drive the evolution of new senses. The main task in SemEval-2020 Task 1 is to identify and evaluate LSC in a set of target words between two text corpora stemming from two different time periods t1 (earlier period) and t2 (later period). The investigated languages are German, English, Swedish and Latin. The task is split into two subtasks: (i) a binary classification task, where it has to be determined whether a target word lost/gained senses between t1 and t2 or not, and (ii) a ranking task, where target words are ranked according to their degree of LSC (a higher rank indicates stronger change). The system presented in this paper is called DiaSense and addresses the LSC tasks by modeling the different senses of a word via contextualized word embeddings using pre-trained BERT (Devlin et al., 2018).1 The binary classification and the ranking task are approached by transferring the measures of change suggested by Schlechtweg et al. (2018), which are originally based on human annotated values for meaning relatedness, to change metrics calculated on the basis of differences between target word embeddings. DiaSense is able to detect LSC in the majority of cases in all four languages (average accuracy 0.728).2 Although the results produced for the ranking task only show a weak correlation with the gold data (Spearman’s ρ = 0.337), DiaSense is able to distinguish between strong and weak …

Content truncated to 500 words