Speakers Fill Lexical Semantic Gaps with Context

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4004–4015, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4004 Speakers Fill Lexical Semantic Gaps with Context Tiago PimentelD Rowan Hall MaudslayD Dami´an Blasi@,S,K Ryan CotterellD,Q DUniversity of Cambridge @Harvard University SMPI SHH KHSE University QETH Z¨urich tp472@cam.ac.uk, rh635@cam.ac.uk, dblasi@fas.harvard.edu, ryan.cotterell@inf.ethz.ch Abstract Lexical ambiguity is widespread in language, allowing for the reuse of economical word forms and therefore making language more ef- ficient. If ambiguous words cannot be disam- biguated from context, however, this gain in efficiency might make language less clear— resulting in frequent miscommunication. For a language to be clear and efficiently encoded, we posit that the lexical ambiguity of a word type should correlate with how much informa- tion context provides about it, on average. To investigate whether this is the case, we oper- ationalise the lexical ambiguity of a word as the entropy of meanings it can take, and pro- vide two ways to estimate this—one which re- quires human annotation (using WordNet), and one which does not (using BERT), making it readily applicable to a large number of lan- guages. We validate these measures by show- ing that, on six high-resource languages, there are significant Pearson correlations between our BERT-based estimate of ambiguity and the number of synonyms a word has in Word- Net (e.g. ρ = 0.40 in English). We then test our main hypothesis—that a word’s lexical ambiguity should negatively correlate with its contextual uncertainty—and find significant correlations on all 18 typologically diverse lan- guages we analyse. This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative. 1 Introduction Linguistic structure and meaning are often underde- termined in the linguistic signal. In an extreme case this can lead to ambiguity: sentences might allow more than one valid syntactic structure, and pro- nouns could corefer to various antecedents. Com- plementarily, linguistic signals can also overdeter- mine some aspect of the intended message—for in- stance, agreement patterns may require redundant marking, and word forms might occupy sparsely 0 5 10 15 20 Contextual Uncertainty (bits) 400 200 0 200 400 Lexical Ambiguity (bits) araf bn fifaeten isidhe kn ml tttrtlptmr yo Figure 1: The relationship between contextual uncertainty—how uncertain a word is given its context—and lexical ambiguity, across a diverse set of languages. populated parts of the phonological space (Harley and Bown, 1998). In a tradition that goes back at least to Zipf, it has been hypothesised that individuals maintain an effi- cient balance between over- and under-specifying an intended message. Such balance is mediated by conflicting pressures for both clarity (the quality that allows the reconstruction of the intended mes- sage), and economy of expression (which allows for inexpensive and rapid encoding of the message in a linguistic signal). A recent instantiation of this idea is that in an efficient language, one expects economical words (which are short or phonotactically simple) to be associated with multiple unrelated meanings, so they can be more widely used (Piantadosi …

Content truncated to 500 words