How (Non-)Optimal is the Lexicon?

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4426–4438 June 6–11, 2021. ©2021 Association for Computational Linguistics 4426 How (Non-)Optimal is the Lexicon? Tiago Pimentel∗,D Irene Nikkarinen∗,D,7 Kyle MahowaldN Ryan CotterellD,Q Dami´an E. Blasi@,9,6 DUniversity of Cambridge 7Yle NUniversity of California, Santa Barbara QETH Z¨urich @Harvard University 9MPI for Evolutionary Anthropology 6HSE University tp472@cam.ac.uk, irene.nikkarinen@gmail.com, mahowald@ucsb.edu ryan.cotterell@inf.ethz.ch, dblasi@fas.harvard.edu Abstract The mapping of lexical meanings to word- forms is a major feature of natural languages. While usage pressures might assign short words to frequent meanings (Zipf’s law of abbreviation), the need for a productive and open-ended vocabulary, local constraints on sequences of symbols, and various other fac- tors all shape the lexicons of the world’s lan- guages. Despite their importance in shaping lexical structure, the relative contributions of these factors have not been fully quantified. Taking a coding-theoretic view of the lexicon and making use of a novel generative statis- tical model, we define upper bounds for the compressibility of the lexicon under various constraints. Examining corpora from 7 typo- logically diverse languages, we use those up- per bounds to quantify the lexicon’s optimality and to explore the relative costs of major con- straints on natural codes. We find that (com- positional) morphology and graphotactics can sufficiently account for most of the complexity of natural codes—as measured by code length. 1 Introduction Communication through language can be modeled under Shannon’s classic communication frame- work (Shannon, 1948). Under this perspective, linguistic utterances are codes—which need to be decoded by a receiver (listener) who is interested in the message (meaning) they encode. Famously, Zipf (1949) posited that language users shape these codes so to accommodate the principle of least effort. The most widely discussed and investigated empirical evidence for this feature is the so-called law of abbreviation, an ostensive negative corre- lation between word frequency and word length (Zipf, 1935; Bentz and Ferrer-i-Cancho, 2016). Communication effort decreases by encoding fre- quent messages in shorter words. ∗Equal contribution 4 5 6 7 8 9 10 11 Code Length Optimal Morph cost of morphological composition cost of morphological composition Morph + Graph Natural cost of inherent randomness Shuffle gain from Zipf's law of abbreviation cost of graphotactics cost of graphotactics Graph Figure 1: The average code length—under our coding schemes—on a representative language (Finnish). The distance between the baselines can be thought of as the cost of each constraint added to the system. This correlation, however, is characteristically modest. There are many instances of short low- frequency words, like wen and jib in English,1 and long frequent words, like happiness and anything. While the lexicon might be shaped by economy of expression, it is clearly not fully optimized for it. There are multiple—possibly competing—reasons why this could be the case. First of all, the sequence of speech sounds, signs, or orthographic characters that serve as building blocks in a language comply with specific rules. These are referred to as phonotactics (in the case …

Content truncated to 500 words