Mining the UK Web Archive for Semantic Change Detection

Proceedings of Recent Advances in Natural Language Processing, pages 1212–1221, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_139 1212 Mining the UK Web Archive for Semantic Change Detection Adam Tsakalidis1, Marya Bazzi1,2,3, Mihai Cucuringu1,3, Pierpaolo Basile5, Barbara McGillivray1,4 1 The Alan Turing Institute, London, United Kingdom 2 University of Warwick, Coventry, United Kingdom 3 University of Oxford, Oxford, United Kingdom 4 University of Cambridge, Cambridge, United Kingdom 5 University of Bari, Bari, Italy {atsakalidis, mbazzi, mcucuringu, bmcgillivray}@turing.ac.uk pierpaolo.basile@uniba.it Abstract Semantic change detection (i.e., identify- ing words whose meaning has changed over time) started emerging as a grow- ing area of research over the past decade, with important downstream applications in natural language processing, historical linguistics and computational social sci- ence. However, several obstacles make progress in the domain slow and diffi- cult. These pertain primarily to the lack of well-established gold standard datasets, resources to study the problem at a fine- grained temporal resolution, and quantita- tive evaluation approaches. In this work, we aim to mitigate these issues by (a) re- leasing a new labelled dataset of more than 47K word vectors trained on the UK Web Archive over a short time-frame (2000- 2013); (b) proposing a variant of Pro- crustes alignment to detect words that have undergone semantic shift; and (c) intro- ducing a rank-based approach for evalu- ation purposes. Through extensive nu- merical experiments and validation, we il- lustrate the effectiveness of our approach against competitive baselines. Finally, we also make our resources publicly available to further enable research in the domain. 1 Introduction Semantic change detection is the task of identi- fying words whose lexical meaning has changed over time. Detecting this temporal variation en- ables historical and social scientists to study cul- tural shifts over time (Michel et al., 2011), but it can also have important implications on the per- formance of models in various NLP tasks, such as sentiment analysis (Lukeˇs and Søgaard, 2018). While early theoretical work on semantic change dates back to the previous century (Bloom- field, 1933), the recent availability of historical datasets has made the computational study of the task feasible (Sandhaus, 2008; Michel et al., 2011; Davies, 2012). Past work has demonstrated that semantic change can manifest over decades (Cook and Stevenson, 2010; Mihalcea and Nas- tase, 2012), years (Yao et al., 2018; Basile and McGillivray, 2018), or even months and weeks (Kulkarni et al., 2015; Tsakalidis et al., 2018). However, important gaps make progress in the field slow. In particular, there is a relative lack of labelled datasets to study the task over a short time-frame, since most known instances of seman- tic change took place over centuries or decades. Furthermore, the evaluation of a semantic change detection model is typically performed by manu- ally inspecting a few examples, which can result in unreliable or even non-measurable performance. Finally, on a methodological front, a common practice to measure the semantic shift of words between consecutive time periods is to calculate their displacement error that results from “align- ing” word vector representations …

Content truncated to 500 words