nlpUP at SemEval-2020 Task 12 : A Blazing Fast System for Offensive Language Detection

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 222–231 Barcelona, Spain (Online), December 12, 2020. 222 TemporalTeller at SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection with Temporal Referencing Jinan Zhou ∗, Jiaxin Li ∗ Department of Computer Science and Engineering The Chinese University of Hong Kong {jinan.zhou, jiaxin.li}@link.cuhk.edu.hk Abstract This paper describes our TemporalTeller system for SemEval Task 1: Unsupervised Lexical Semantic Change Detection (Schlechtweg et al., 2020). We develop a unified framework for the common semantic change detection pipelines including preprocessing, learning word embeddings, calculating vector distances and determining threshold. We also propose Gamma Quantile Threshold to distinguish between changed and stable words. Based on our system, we conduct a comprehensive comparison among BERT, Skip-gram, Temporal Referencing and alignment-based methods. Evaluation results show that Skip-gram with Temporal Referencing achieves the best performance of 66.5% classification accuracy and 51.8% Spearman’s Ranking Correlation. 1 Introduction Task 1 in SemEval 2020 is Unsupervised Lexical Semantic Change Detection. Semantic change is defined as the changes in lexical meaning rather than grammatical usage of a language symbol (Bloomfield, 1933). It has become an increasingly common phenomenon under the influence of linguistic habits, social environments and many other factors. For example, the word gay previously meant being happy but now mostly means homosexual. Those changes are indicators of temporal ideologies as well as language development trends. Therefore, studying the evolution of word meanings can assist the research in human language, historical information retrieval and arguably other disciplines. Recently, there have been many studies on this topic. One direction is word level change detection. Those approaches seek to represent each word as a single vector that reflects its overall meanings. Some works (Sagi et al., 2009; Gulordava and Baroni, 2011; Rodda et al., 2016; Kahmann et al., 2017) use co-occurrence matrices with SVD. Others (Kim et al., 2014; Rosenfeld and Erk, 2018; Hamilton et al., 2016a) adopt neural embeddings such as Skip-gram with negative sampling (SGNS) (Mikolov et al., 2013a). However, those word embeddings need to be aligned to the same vector space, which is shown to introduce noise (Dubossarsky et al., 2019). Another direction is detecting semantic changes from the sense level. These models investigate the different meanings of words, thus can compare in a fine grained manner. In this paper, we will focus on word level change detection. We spotted two problems among those related works. First, no classification mechanisms have yet been discussed. All papers calculate the lexical semantic change, but none has proposed a systematic approach to classify between changed and stable words. The second problem is that the models may be potentially out-of-date. While the leaderboards of many NLP tasks such as reading comprehension are continuously refreshed by new models like Pre-training of Deep Bidirectional Transformers (BERT) (Devlin et al., 2018), the methods adopted in this topic are relatively conventional. Though it is not necessarily a problem, we are curious to know whether the latest models could bring extra benefits. Aiming at the first problem, we propose Gamma …

Content truncated to 500 words