Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 66–76, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 66 Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings Philippa Shoemark∗♠♦, Farhana Ferdousi Liza∗♠♥, Dong Nguyen♠♦, Scott A. Hale♠†, Barbara McGillivray♠‡ ♠The Alan Turing Institute, UK, ♦School of Informatics, University of Edinburgh, UK, ♥School of Computing, University of Kent, UK, †Oxford Internet Institute, University of Oxford, UK, ‡Faculty of Modern and Medieval Languages, University of Cambridge, UK p.j.shoemark@ed.ac.uk, fl207@kent.ac.uk, dnguyen@turing.ac.uk, scott.hale@oii.ox.ac.uk, bmcgillivray@turing.ac.uk Abstract Word embeddings are increasingly used for the automatic detection of semantic change; yet, a robust evaluation and systematic com- parison of the choices involved has been lack- ing. We propose a new evaluation framework for semantic change detection and find that (i) using the whole time series is preferable over only comparing between the first and last time points; (ii) independently trained and aligned embeddings perform better than continuously trained embeddings for long time periods; and (iii) that the reference point for comparison matters. We also present an analysis of the changes detected on a large Twitter dataset spanning 5.5 years. 1 Introduction Semantic change, i.e., the change in the meanings of words, is inherent in language. A new mean- ing for a word can be added to the original one, become more or less prevalent, or even replace a former meaning (see Koch, 2016). An exam- ple is lit, which has gained a new sense of ‘excit- ing’ or ‘awesome’, via the extension of its long- established use as slang for ‘intoxicated’ to de- scribe the vibrant environment in which acts of be- coming intoxicated often occur.1 Automatically measuring semantic change can discover changes that would not be apparent from manual inspection. It can also facilitate the inves- tigation of mechanisms driving semantic changes, e.g., how these changes are affected by language- internal and social factors. Moreover, there are direct benefits to applications, such as the detec- tion of meaning shifts in polarized words to update sentiment lexicons and the detection of emerging word meanings to update dictionaries. ∗*These authors contributed equally to the study. 1“What Does ‘Lit’ Mean?” Retrieved from https://www.merriam-webster.com/words-at-play/lit- meaning-origin. Accessed 2018-10-05. Word embeddings are increasingly used for au- tomatic semantic change detection (Kutuzov et al., 2018). Words are mapped to low-dimensional vec- tors, and the semantic change of a word is then measured by comparing its vectors across time pe- riods. Although word embeddings have emerged as one of the most popular approaches to measur- ing semantic change, researchers are faced with various decisions, including whether to train em- beddings independently or continuously, which metric to use to measure change between two time periods, and which ranking approach to use for comparing semantic change candidates. A major challenge in developing semantic change detection systems is obtaining ground truth data (Kutuzov et al., 2018), which has so far pre- vented a systematic …

Content truncated to 500 words