Dynamic Generative model for Diachronic Sense Emergence Detection

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1362–1373, Osaka, Japan, December 11-17 2016. Dynamic Generative model for Diachronic Sense Emergence Detection Martin Emms Dept of Computer Science Trinity College, Dublin Ireland martin.emms@scss.tcd.ie Arun Jayapal Dept of Computer Science Trinity College, Dublin Ireland jayapala@scss.tcd.ie Abstract As time passes words can acquire meanings they did not previously have, such as the ‘twitter post’ usage of ‘tweet’. We address how this can be detected from time-stamped raw text. We propose a generative model with senses dependent on times and context words dependent on senses but otherwise eternal, and a Gibbs sampler for estimation. We obtain promising parameter estimates for positive (resp. negative) cases of known sense emergence (resp non-emergence) and adapt the ‘pseudo-word’ technique (Sch¨utze, 1998) to give a novel further evaluation via ‘pseudo-neologisms’. The question of ground-truth is also addressed and a technique proposed to locate an emergence date for evaluation purposes. 1 Introduction It is widely noted that a single word can have several senses. The diachronic aspect of this is that the set of senses possessed by a word changes over time. (1a) and (1b) below illustrate this: (a) she was a gay little soul, enjoying everything and always trilling with laughter (1905) (b) applying heightened scrutiny to discrimination against gay men and lesbians (1990) (a′) sie war ein Homosexuell kleine Seele, alles zu genießen und immer rollen vor Lachen (1) (1a), from 1905, illustrates a ‘being happy’ sense of gay, while (1b), dating from 1990, illustrates a ‘homosexual’ sense, a sense which the word did not possess in 1905, and came to possess at some time since. The advent of a new sense for an existing form is sometimes called a semantic neologism (Tournier, 1985), in contrast to the simpler formal neologism, where simply a new form arrives (eg. selfie). The concern of this paper is to propose an unsupervised algorithm for detecting semantic neolo- gisms, an algorithm which can be given time-stamped but plain-text examples of a particular word and detect whether (and when) the word gained a sense. Information about such lexical change could be useful to NLP tasks. For example, if a SMT system is trained on data from particular times and is to be applied to texts from different times, either later or earlier, advance warning of sense changes could be of use. To illustrate, (1a′) gives the English→German translation via Google translate1 of (1a), mistranslating the 1905 usage of gay as Homosexuell, probably due to the newer sense predominating in training data. We will propose a diachronic sense model where a target’s sense is conditioned on time and the context words are conditioned just on the target’s sense, and not the time. We use the Google n-gram data set (Michel et al., 2011) which provides time-stamped data but no sense information and develop a Gibbs sampling algorithm (Gelfand and Smith, 1990) to estimate the parameters in an unsupervised fashion. We will show that the algorithm is able …

Content truncated to 500 words