A Bayesian Model of Diachronic Meaning Change

A Bayesian Model of Diachronic Meaning Change Lea Frermann and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB l.frermann@ed.ac.uk, mlap@inf.ed.ac.uk Abstract Word meanings change over time and an au- tomated procedure for extracting this infor- mation from text would be useful for histor- ical exploratory studies, information retrieval or question answering. We present a dy- namic Bayesian model of diachronic meaning change, which infers temporal word represen- tations as a set of senses and their prevalence. Unlike previous work, we explicitly model language change as a smooth, gradual pro- cess. We experimentally show that this model- ing decision is beneficial: our model performs competitively on meaning change detection tasks whilst inducing discernible word senses and their development over time. Application of our model to the SemEval-2015 temporal classification benchmark datasets further re- veals that it performs on par with highly op- timized task-specific systems. 1 Introduction Language is a dynamic system, constantly evolv- ing and adapting to the needs of its users and their environment (Aitchison, 2001). Words in all lan- guages naturally exhibit a range of senses whose dis- tribution or prevalence varies according to the genre and register of the discourse as well as its historical context. As an example, consider the word cute which according to the Oxford English Dictionary (OED, Stevenson 2010) first appeared in the early 18th century and originally meant clever or keen- witted.1 By the late 19th century cute was used in 1Throughout this paper we denote words in true type, their senses in italics, and sense-specific context words as {lists}. the same sense as cunning. Today it mostly refers to objects or people perceived as attractive, pretty or sweet. Another example is the word mouse which initially was only used in the rodent sense. The OED dates the computer pointing device sense of mouse to 1965. The latter sense has become par- ticularly dominant in recent decades due to the ever- increasing use of computer technology. The arrival of large-scale collections of historic texts (Davies, 2010) and online libraries such as the Internet Archive and Google Books have greatly facilitated computational investigations of language change. The ability to automatically detect how the meaning of words evolves over time is potentially of significant value to lexicographic and linguistic research but also to real world applications. Time- specific knowledge would presumably render word meaning representations more accurate, and benefit several downstream tasks where semantic informa- tion is crucial. Examples include information re- trieval and question answering, where time-related information could increase the precision of query disambiguation and document retrieval (e.g., by re- turning documents with newly created senses or fil- tering out documents with obsolete senses). In this paper we present a dynamic Bayesian model of diachronic meaning change. Word mean- ing is modeled as a set of senses, which are tracked over a sequence of contiguous time intervals. We infer temporal meaning representations, consisting of a word’s senses (as a probability …

Content truncated to 500 words