Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 457–470 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 457 Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change Haim Dubossarsky♠ Simon Hengchen♦ Nina Tahmasebi♣ Dominik Schlechtweg♥ ∗ ♠ Language Technology Lab, University of Cambridge ♦ COMHIS, University of Helsinki ♣ Department of Swedish, University of Gothenburg ♥ Institute for Natural Language Processing, University of Stuttgart hd423@cam.ac.uk simon.hengchen@helsinki.fi nina.tahmasebi@gu.se schlecdk@ims.uni-stuttgart.de Abstract State-of-the-art models of lexical semantic change detection suffer from noise stemming from vector space alignment. We have empiri- cally tested the Temporal Referencing method for lexical semantic change and show that, by avoiding alignment, it is less affected by this noise. We show that, trained on a diachronic corpus, the skip-gram with negative sampling architecture with temporal referencing outper- forms alignment models on a synthetic task as well as a manual testset. We introduce a prin- cipled way to simulate lexical semantic change and systematically control for possible biases. 1 Introduction These past years have seen the rise of computa- tional methods to detect, track, qualify, and quan- tify how a word’s sense – or senses – change over time. These tasks are critical challenges that are relevant to a range of NLP fields, including the study of historical semantic change. The success- ful outcome of semantic change detection is rel- evant to any diachronic textual analysis, includ- ing machine translation or normalization of his- torical texts (Tjong Kim Sang et al., 2017), the de- tection of cultural semantic shifts (Kutuzov et al., 2017) or applications in digital humanities (Tah- masebi and Risse, 2017a). However, currently, the best-performing models (Hamilton et al., 2016b; Kulkarni et al., 2015; Schlechtweg et al., 2019) require a complex alignment procedure and have been shown to suffer from biases (Dubossarsky et al., 2017). This exposes them to various sources of noise influencing their predictions; a fact which has long gone unnoticed because of the lack of standard evaluation procedures in the field. We examine the modeling approach of Tempo- ral Referencing (TR) which avoids post hoc align- ∗The order has been randomly determined and all authors contributed equally to this work. ment and is applicable to any vector space learning technique. We show that it (i) is less affected by noise and (ii) clearly outperforms state-of-the-art alignment models on a synthetic change detection task. The task is based on data from a synchronic corpus into which we artificially inject lexical se- mantic change (LSC) in a controlled and semanti- cally principled way. We further evaluate the mod- els on a manual testset of diachronic LSC and ex- amine their properties. In this paper, we focus on skip-gram with neg- ative sampling (SGNS) models (Mikolov et al., 2013) and PPMI (Levy et al., 2015) and make use of TR to share context information across time pe- riods, while learning individual embeddings for a target word in each time period. We evaluate mod- els in two ways: …

Content truncated to 500 words