ECNU at SemEval-2020 Task 7: Assessing Humor in Edited News Headlines Using BiLSTM with Attention

Proceedings of the 14th International Workshop on Semantic Evaluation, pages 112–118 Barcelona, Spain (Online), December 12, 2020. 112 SenseCluster at SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection Amaru Cuba Gyllensten RISE amaru.cuba.gyllensten@ri.se Evangelia Gogoulou RISE evangelia.gogoulou@ri.se Ariel Ekgren RISE ariel.ekgren@ri.se Magnus Sahlgren RISE magnus.sahlgren@ri.se Abstract We (Team Skurt) propose a simple method to detect lexical semantic change by clustering contextualized embeddings produced by XLM-R, using K-Means++. The basic idea is that contextualized embeddings that encode the same sense are located in close proximity in the embedding space. Our approach is both simple and generic, but yet performs relatively well in both sub-tasks of SemEval-2020 Task 1. We hypothesize that the main shortcoming of our method lies in the simplicity of the clustering method used. 1 Introduction The meaning of a word can vary not only with context, but also with time. The former phenomenon is commonly known as context-sensitivity, or, if the variation is of a more categorical nature, polysemy, whereas the latter phenomenon is captured by the term diachronic semantic drift (Kutuzov et al., 2018), or alternatively lexical semantic change (Schlechtweg et al., 2020). As an example, a term such as “beautiful” has one main meaning, but will nonetheless imply slightly different things depending on its context of use; “suit” on the other hand has several distinct meanings (e.g. as a verb or as a noun), while “mouse” has acquired a completely new meaning with the introduction of computer hardware. Of course, the distinction between context-sensitivity and polysemy is anything but clear-cut; this is a slippery theoretical slope, on which it is best to tread lightly. Even so, enabling the detection of such diachronic lexical semantic change across time could accelerate research in historical linguistics (Szymanski, 2017), and also initiate the development of decision-making systems that exploit diachronically shifting information (Rosin et al., 2017). The backbone of a lexical semantic change detection system is word embeddings, which represent the meaning (or at least the use) of words. Different systems rely on various types of language models, nowadays predominantly distributional in nature. There is a comparably rich literature on distributional approaches to modeling diachronic semantic drift; examples include Sagi et al. (2008) Hamilton et al. (2016), and Yao et al. (2018). More complete overviews of existing diachronic semantic shift detection techniques is provided in Tahmasebi et al. (2018) and Kutuzov et al. (2018). Contextualized language models constitute a recent breakthrough in the field of NLP (Devlin et al., 2018; Radford et al., 2018), by virtue of their ability to provide embeddings that are sensitive to a specific context of use, which is different from standard word embeddings that aggregate all of a word’s contexts into one global representation. Another way of characterizing this difference is to say that contextualized language models provide token-based representations, while standard word embeddings are type-based. Motivated by the success of contextualized language models for handling polysemy and context-sensitivity, we investigate how such embeddings can be used to model diachronic semantic change; we use a …

Content truncated to 500 words