A Bayesian Model of Grounded Color Semantics

A Bayesian Model of Grounded Color Semantics Brian McMahan Rutgers University brian.mcmahan@rutgers.edu Matthew Stone Rutgers University matthew.stone@rutgers.edu Abstract Natural language meanings allow speakers to encode important real-world distinctions, but corpora of grounded language use also re- veal that speakers categorize the world in different ways and describe situations with different terminology. To learn meanings from data, we therefore need to link underly- ing representations of meaning to models of speaker judgment and speaker choice. This paper describes a new approach to this prob- lem: we model variability through uncertainty in categorization boundaries and distributions over preferred vocabulary. We apply the ap- proach to a large data set of color descrip- tions, where statistical evaluation documents its accuracy. The results are available as a Lexicon of Uncertain Color Standards (LUX), which supports future efforts in grounded lan- guage understanding and generation by prob- abilistically mapping 829 English color de- scriptions to potentially context-sensitive re- gions in HSV color space. 1 Introduction To ground natural language semantics in real-world data at large scale requires researchers to confront the vocabulary problem (Furnas et al., 1987). Much of what people say falls in a long tail of increas- ingly infrequent and specialized items. Moreover, the choice of how to categorize and describe real- world data varies across people. We can’t account for this complexity by deriving one definitive map- ping between words and the world. We see this complexity already in free text de- scriptions of color patches. English has fewer than Hue 0 1 2 3 4 5 6 Entropy (bits) Figure 1: A visualization of the variability of the de- scriptions used to name colors within small bins of color space. For each Hue value, the entropy values for each bin along the Saturation and Value dimensions are grouped and plotted as box plots. The dotted line cor- responds to a random choice out of fourteen items and to the perplexity of a histogram model trained on the corpus. a dozen basic color words (Berlin, 1991), but peo- ple’s descriptions of colors are much more variable than this would suggest. Measured on the corpus described in Section 4.1, there’s an average of 3.845 bits of information in a color description given the color it describes—comparable to rolling a 14-sided die. Figure 1 summarizes the data and plots the en- tropy of descriptions encountered within small bins of color space. The bins are aggregated over the Sat- uration and Value dimensions and indexed on the x- axis by the Hue dimension. There’s little reason to think that this variability conceals consistent mean- ings. In formal semantics, one of the hallmarks of vague language is that speakers can make it more precise in alternative, incompatible ways (Barker, 2002). We see this in practice as well, for exam- ple with the image of Figure 2, where subjects com- 103 Transactions of the Association for Computational Linguistics, vol. 3, pp. 103–115, 2015. Action Editor: Lillian Lee. Submission batch: 11/2014; Published 2/2015. c⃝2015 Association for Computational Linguistics. …

Content truncated to 500 words