Intentional Context in Situated Natural Language Learning

Proceedings of the 9th Conference on Computational Natural Language Learning (CoNLL), pages 104–111, Ann Arbor, June 2005. c⃝2005 Association for Computational Linguistics Intentional Context in Situated Natural Language Learning Michael Fleischman and Deb Roy Cognitive Machines The Media Laboratory Massachusetts Institute of Technology mbf@mit.edu, dkroy@media.mit.edu Abstract Natural language interfaces designed for situationally embedded domains (e.g. cars, videogames) must incorporate knowledge about the users’ context to address the many ambiguities of situated language use. We introduce a model of situated language acquisition that operates in two phases. First, intentional context is represented and inferred from user actions using probabilistic context free grammars. Then, utterances are mapped onto this representation in a noisy channel framework. The acquisition model is trained on unconstrained speech collected from subjects playing an interactive game, and tested on an understanding task. 1 Introduction As information technologies move off of our desktops and into the world, the need for Natural Language Processing (NLP) systems that exploit information about the environment becomes increasingly apparent. Whether in physical environments (for cars and cell phones) or in virtual ones (for videogames and training simulators), applications are beginning to demand language interfaces that can understand unconstrained speech about constrained domains. Unlike most text-based NLP research, which focuses on open-domain problems, work we refer to as situated NLP focuses on improving language processing by exploiting domain-specific information about the non-linguistic situational context of users’ interactions. For applications where agents interact in shared environments, such information is critical for successful communication. Previous work in situated NLP has focused on methods for grounding the meaning of words in physical and virtual environments. The motivation for this work comes from the inability of text- based NLP technologies to offer viable models of semantics for human computer interaction in shared environments. For example, imagine a situation in which a human user is interacting with a robotic arm around a table of different colored objects. If the human were to issue the command “give me the blue one,” both the manually-coded (Lenat, 1995; Fellbaum, 1998) and statistical models (Manning and Schutze, 2000) of meaning employed in text-based NLP are inadequate; for, in both models, the meaning of a word is based only on its relations to other words. However, in order for the robot to successfully “give me the blue one,” it must be able to link the meaning of the words in the utterance to its perception of the environment (Roy, Hsiao, & Mavridis, 2004). Thus, recent work on grounding meaning has focused on how words and utterances map onto physical descriptions of the environment: either in the form of perceptual representations (Roy, in press, Siskind, 2001, Regier, 1996) or control schemas (Bailey, 1997 Narayanan, 1999).1 While such physical descriptions are useful representations for some classes of words (e.g., colors, shapes, physical movements), they are insufficient for more abstract language, such as that which denotes intentional action. This insufficiency stems from the fact that intentional actions (i.e. actions performed with the purpose of achieving a goal) are …

Content truncated to 500 words