Quantifying Cognitive Factors in Lexical Decline

Quantifying Cognitive Factors in Lexical Decline David Francis1 Ella Rabinovich1 Farhan Samir1 David Mortensen2 Suzanne Stevenson1 1Department of Computer Science, University of Toronto, Canada 2Language Technologies Institute, Carnegie Mellon University, USA {dfrancis, ella, fsamir, suzanne}@cs.toronto.edu dmortens@cs.cmu.edu Abstract We adopt an evolutionary view on language change in which cognitive factors (in addition to social ones) affect the fitness of words and their success in the linguistic ecosystem. Spe- cifically, we propose a variety of psycholin- guistic factors—semantic, distributional, and phonological—that we hypothesize are predic- tive of lexical decline, in which words greatly decrease in frequency over time. Using his- torical data across three languages (English, French, and German), we find that most of our proposed factors show a significant difference in the expected direction between each curated set of declining words and their matched stable words. Moreover, logistic regression analyses show that semantic and distributional factors are significant in predicting declining words. Further diachronic analysis reveals that de- clining words tend to decrease in the diversity of their lexical contexts over time, gradually narrowing their ‘ecological niches’. 1 Introduction Many researchers, from Schleicher (1863) up to the present (Croft, 2000; Oudeyer and Kaplan, 2007; Atkinson et al., 2008; Thanukos, 2008; Turney and Mohammad, 2019), have drawn analo- gies between biological evolution and the evolu- tion of languages—their structure, their semantics, and their lexicons. Lexically speaking, as Schle- icher first pointed out, diachrony can be viewed as a struggle for survival by individual words whose propagation into future generations is contingent on their continued fitness for one or more niches in the ecology of the speech community—as determined by a host of factors. Here we study the question of lexical decline—a gradual decrease in frequency and ultimate obsolescence of words. What explains that the word ‘amusements’ has declined in the last 200 years, but ‘foundations’ has not, as shown in Figure 1 (along with other similar pairs)? Social factors clearly play a role, as changes in culture and technology may lead words to fall in and out of use. But cognitive and linguistic factors also influence lexical survival (e.g., Vejdemo and H¨orberg, 2016). Words that are semantically similar to many other words may come to be used less because of intense compe- tition in the cognitive process of lexical access (Chen and Mirman, 2012). Words that can occupy many niches, distributionally speaking, should have better chances of being learned and used, and therefore perpetuated, than words that are confined to a narrow range of contexts or senses (Altmann et al., 2011; Stewart and Eisenstein, 2018). On the other hand, words that are pho- nologically very different from other words may suffer because they are more difficult to access, sitting as they do at the formal fringes of the men- tal lexicon (Edwards et al., 2004). In other words, we suggest that semantic, distributional, and pho- nological factors all play a role in the natural selection of words. While attention to predicting which words will emerge, live, and die goes back to Schleicher, …

Content truncated to 500 words