Quantity, Contrast, and Convention in Cross-Situated Language Comprehension

Proceedings of the 19th Conference on Computational Language Learning, pages 226–236, Beijing, China, July 30-31, 2015. c⃝2015 Association for Computational Linguistics Quantity, Contrast, and Convention in Cross-Situated Language Comprehension Ian Perera and James F. Allen University of Rochester, Department of Computer Science, Rochester, NY 14627 USA Institute for Human and Machine Cognition, Pensacola, FL 32502 USA {iperera,jallen}@ihmc.us Abstract Typically, visually-grounded language learning systems only accept feature data about objects in the environment that are explicitly mentioned, whether through annotation labels or direct ref- erence through natural language. We show that when objects are described ambiguously using natural language, a system can use a combination of the pragmatic principles of Contrast and Conventionality, and multiple-instance learning to learn from ambiguous exam- ples in an online fashion. Applying child language learning strategies to visual learning enables more effective learning in real-time environments, which can lead to enhanced teaching interactions with robots or grounded systems in multi-object environments. 1 Introduction As opposed to the serial nature of labeled data presented to a machine learning classifier, chil- dren and robots “in the wild” must learn object names and attributes like color, size, and shape while being surrounded by a number of stimuli and possible referents. When a child hears “the red ball”, they must first identify the object mentioned, then use existing knowledge to identify that “red” and “ball” are distinct concepts, and over time, learn that objects called “red” share some sim- ilarity in color while objects called “ball” share some similarity in shape. Learning for them there- fore requires both identification and establishing joint attention with the speaker before assigning a label to an object, while also applying other language learning strategies to narrow down the search space of possible referents, as illustrated by Quine’s “gavagai” problem (1964). Trying to learn attributes and objects without non-linguistic cues such as pointing and gaze might seem an insurmountable challenge. Yet a child experiences many such situations and can nevertheless learn grounded concepts over time. Fortunately, adult speakers tend to understand the limitation of these cues in certain situations and adjust their speech in accordance to Grice’s Maxim of Quantity when referring to objects : be only as informative as necessary (Grice, 1975). We therefore treat the language describing a par- ticular object in a scene as an expression of an it- erative process, where the speaker is attempting to guide the listener towards the referent in a way that avoids both ambiguity and unnecessary verbosity. Language learners additionally make use of the pragmatic assumptions of Conventionality, that speakers agree upon the meaning of a word, and Contrast, that different words have different mean- ings (Clark, 2009). The extension of these princi- ples to grounded language learning yields the as- sumptions that the referents picked out by a refer- ring expression will have some similarity (percep- tual in our domain), and will be dissimilar com- pared to objects not included in the reference. Children will eventually generalize learned con- cepts or accept synonyms in a …

Content truncated to 500 words