Infinite SCAN: An Infinite Model of Diachronic Semantic Change

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1605 - 1616 December 7-11, 2022 ©2022 Association for Computational Linguistics Infinite SCAN: An Infinite Model of Diachronic Semantic Change Seiichi Inoue1, Mamoru Komachi1, Toshinobu Ogiso2, Hiroya Takamura3 and Daichi Mochihashi4 1Tokyo Metropolitan University 2The National Institute for Japanese Language and Linguistics 3The National Institute of Advanced Industrial Science and Technology 4The Institute of Statistical Mathematics inoue-seiichi@ed.tmu.ac.jp komachi@tmu.ac.jp togiso@ninjal.ac.jp takamura.hiroya@aist.go.jp daichi@ism.ac.jp Abstract In this study, we propose a Bayesian model that can jointly estimate the number of senses of words and their changes through time. The model combines a dynamic topic model on Gaussian Markov random fields (Frermann and Lapata, 2016) with a logistic stick-breaking process that realizes the Dirichlet process. In the experiments, we evaluated the proposed model in terms of interpretability, accuracy in estimating the number of senses, and track- ing their changes using both artificial data and real data. We quantitatively verified that the model behaves as expected through evaluation using artificial data. Using the CCOHA cor- pus, we showed that our model outperforms the baseline model and investigated the semantic changes of several well-known target words. 1 Introduction Words exhibit a range of senses depending on the context in which they are used. These senses can also change over time (Blank and Koch, 1999; Aitchison, 2001). For example, the word cute ap- peared in the early 18th century, which originally meant clever or keen-witted. By the late 19th cen- tury it signified cunning, and today, cute means attractive, pretty, or sweet (Stevenson, 2010; Fr- ermann and Lapata, 2016). Automatically captur- ing these semantic changes is an academic contri- bution to the fields of lexicology and linguistics (Voyles, 1973; Williams, 1976). In recent years, many methods have been pro- posed for the detection of semantic changes us- ing distributional methods (Kutuzov et al., 2018). They include word embedding-based methods with alignment of word embedding spaces at different times (Kim et al., 2014; Kulkarni et al., 2015; Hamilton et al., 2016b), without alignment (Yao et al., 2018; Dubossarsky et al., 2019; Aida et al., 2021), and using probabilistic frameworks (Bam- ler and Mandt, 2017; Rudolph and Blei, 2018). Word embedding-based methods describe semantic change by changes of surrounding words in se- mantic space. However, the learned embeddings themselves cannot account for the existence of each sense and its relative importance. In contrast, sev- eral methods have addressed these issues using a topic model architecture (Frermann and Lapata, 2016; Emms and Jayapal, 2016). These proba- bilistic models estimate the latent senses explicitly and consider their changes, unlike word embed- ding models. However, these models have a criti- cal problem in that the number of senses is given and fixed, even though it will vary for each word, which harms the modeling of semantic change. In addition, the number of senses is rarely apparent beforehand, thus it is difficult to set it a priori. A re- cent method of clustering contextualized word em- beddings obtained from BERT …

Content truncated to 500 words