Outta Control: Laws of Semantic Change and Inherent Biases in Word Representation Models

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1136–1145 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Outta Control: Laws of Semantic Change and Inherent Biases in Word Representation Models Haim Dubossarsky1, Eitan Grossman2 and Daphna Weinshall3 1 Edmond and Lily Safra Center for Brain Sciences 2 Department of Linguistics 3 School of Computer Science and Engineering The Hebrew University of Jerusalem, 91904 Jerusalem, Israel haim.dub@gmail.com, {eitan.grossman,daphna}@mail.huji.ac.il Abstract This article evaluates three proposed laws of semantic change. Our claim is that in order to validate a putative law of seman- tic change, the effect should be observed in the genuine condition but absent or re- duced in a suitably matched control condi- tion, in which no change can possibly have taken place. Our analysis shows that the effects reported in recent literature must be substantially revised: (i) the proposed neg- ative correlation between meaning change and word frequency is shown to be largely an artefact of the models of word represen- tation used; (ii) the proposed negative cor- relation between meaning change and pro- totypicality is shown to be much weaker than what has been claimed in prior art; and (iii) the proposed positive correlation between meaning change and polysemy is largely an artefact of word frequency. These empirical observations are corrob- orated by analytical proofs that show that count representations introduce an inher- ent dependence on word frequency, and thus word frequency cannot be evaluated as an independent factor with these repre- sentations. 1 Introduction The increasing availability of digitized histori- cal corpora, together with newly developed tools of computational analysis, make the quantitative study of language change possible on a larger scale than ever before. Thus, many important ques- tions may now be addressed using a variety of NLP tools that were originally developed to study synchronic similarities between words. This has catalyzed the evolution of an exciting new field of historical distributional semantics, which has yielded findings that inform our understanding of the dynamic structure of language (Sagi et al., 2009; Wijaya and Yeniterzi, 2011; Mitra et al., 2014; Hilpert and Perek, 2015; Frermann and La- pata, 2016; Dubossarsky et al., 2016). Recent research has even proposed laws of change that predict the conditions under which the meaning of words is likely to change (Dubossarsky et al., 2015; Xu and Kemp, 2015; Hamilton et al., 2016). This is an important development, as traditional historical linguistics has generally been unable to provide predictive models of semantic change. However, these preliminary results should be addressed with caution. To date, analyses of changes in words’ meanings have relied on the comparison of word representations at different points in time. Thus any proposed change in meaning is contingent on a particular model of word representation and the method used to mea- sure change. Distributional semantic models typi- cally count words and their co-occurrence statis- tics (explicit models) or predict the embedding contexts of words (implicit models). In this paper, we show that the choice …

Content truncated to 500 words