On Homophony and Rényi Entropy

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8284–8293 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 8284 On Homophony and R´enyi Entropy Tiago Pimentel Clara Meister Simone Teufel Ryan Cotterell University of Cambridge ETH Z¨urich tp472@cam.ac.uk clara.meister@inf.ethz.ch sht25@cl.cam.ac.uk ryan.cotterell@inf.ethz.ch Abstract Homophony’s widespread presence in natural languages is a controversial topic. Recent theories of language optimality have tried to justify its prevalence, despite its negative ef- fects on cognitive processing time; e.g., Pi- antadosi et al. (2012) argued homophony en- ables the reuse of efficient wordforms and is thus beneficial for languages. This hypothe- sis has recently been challenged by Trott and Bergen (2020), who posit that good wordforms are more often homophonous simply because they are more phonotactically probable. In this paper, we join in on the debate. We first propose a new information-theoretic quantifi- cation of a language’s homophony: the sam- ple R´enyi entropy. Then, we use this quan- tification to revisit Trott and Bergen’s claims. While their point is theoretically sound, a spe- cific methodological issue in their experiments raises doubts about their results. After address- ing this issue, we find no clear pressure either towards or against homophony—a much more nuanced result than either Piantadosi et al.’s or Trott and Bergen’s findings. 1 Introduction Ambiguity is a hallmark of human language, and is present at all levels of linguistic structure. Both the causes and resulting effects of ambiguity are topics which have sparked much debate. For example, while some claim it has a beneficial impact on communication, others have taken it as a sign of inefficiency. In this work, we contribute to the debate surrounding a specific form of lexical ambiguity in which a wordform shares multiple unrelated meanings: homophony.1 While the quantitative study of homophony dates back to (at least) Zipf (1949), recently, Trott and Bergen (2020) proposed a new explanation for the higher rate of homophony amongst good, i.e. short or phonotactically well-formed, wordforms: 1In English, one example is the word /naIt/ which, out of context, could mean either KNIGHT or NIGHT. if words were sampled i.i.d. from a phonotactic distribution, then good wordforms would simply be sampled more often. This implies there is no pressure favouring homophony in these words for the sake of efficiency, which directly opposes Pi- antadosi et al.’s (2012) hypothesis. In fact, Trott and Bergen go further, relying on their experimen- tal results to argue that “homophony may even be selected against in real languages.”2 In this work, we join this debate by proposing a novel quantification of a language’s homophony: the sample R´enyi entropy3—defined as the negative log likelihood (or surprisal) that two instances in an M-sized sample take on the same value. When measured on an observed lexicon, this measure holistically captures the chance that two wordforms coincide, i.e., that they are homophones, providing a new means to test whether lexicons have a pressure in favour or against homophony. Further, we revisit Trott and Bergen’s claims. Whilst their theoretical arguments are sound, …

Content truncated to 500 words