Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 292–301 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 292 Comparing Models of Associative Meaning: An Empirical Investigation of Reference in Simple Language Games Judy Hanwen Shen Matthias Hofer Bjarke Felbo Roger Levy Massachusetts Institute of Technology 77 Massachusetts Avenue Cambridge, MA 02139 {judyshen, mhofer, felbo, rplevy}@mit.edu Abstract Simple reference games (Wittgenstein, 1953) are of central theoretical and empirical im- portance in the study of situated language use. Although language provides rich, com- positional truth-conditional semantics to fa- cilitate reference, speakers and listeners may sometimes lack the overall lexical and cog- nitive resources to guarantee successful ref- erence through these means alone. However, language also has rich associational structures that can serve as a further resource for achiev- ing successful reference. Here we investigate this use of associational information in a set- ting where only associational information is available: a simplified version of the popu- lar game Codenames. Using optimal experi- ment design techniques, we compare a range of models varying in the type of associative information deployed and in level of prag- matic sophistication against human behavior. In this setting we find that listeners’ behav- ior reflects direct bigram collocational associ- ations more strongly than word-embedding or semantic knowledge graph-based associations and that there is little evidence for pragmat- ically sophisticated behavior by either speak- ers or listeners of the type that might be pre- dicted by recursive-reasoning models such as the Rational Speech Acts theory. These results shed light on the nature of the lexical resources that speakers and listeners can bring to bear in achieving reference through associative mean- ing alone. 1 Introduction In his 1953 book Philosophical Investigations, Wittgenstein makes the argument for studying simple reference games to learn about the nature of language (Wittgenstein, 1953). Various ap- plications of this idea in different fields, includ- ing linguistics (Pietarinen, 2007), cognitive sci- ence (Frank and Goodman, 2012), artificial in- telligence (Lazaridou et al., 2016), and behavior- based robotics (Steels, 1997) have validated this fundamental insight and demonstrated the theo- retical and empirical importance of studying lan- guage learning and use in simplified contexts. Here we describe a novel framework that uses a simple reference game to study the semantic re- sources speakers and listeners use to facilitate ref- erence. In particular, placing strong constraints on word choice and modes of interaction allows us to better isolate specific aspects that contribute to- wards the complexity of natural language seman- tics. Language provides a multitude of different resources for its users to cooperatively achieve reference. In particular, language provides truth- conditional semantic structures. These informa- tion structures are characterized in terms of their logical truth conditions and can be precisely stated using formal logic. Across many cases, however, successful reference cannot be guaranteed through these means alone. Another possible source of se- mantic information are associative resources (e.g. the meaning associations of ‘nurse’ with ‘female nurse’ rather than ‘male nurse’). …

Content truncated to 500 words