Incremental Generation of Spatial Referring Expressions in Situated Dialog

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 1041–1048, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Incremental generation of spatial referring expressions in situated dialog∗ John D. Kelleher Dublin Institute of Technology Dublin, Ireland john.kelleher@comp.dit.ie Geert-Jan M. Kruijff DFKI GmbH Saarbr¨ucken, Germany gj@dfki.de Abstract This paper presents an approach to incrementally generating locative expressions. It addresses the is- sue of combinatorial explosion inherent in the con- struction of relational context models by: (a) con- textually defining the set of objects in the context that may function as a landmark, and (b) sequenc- ing the order in which spatial relations are consid- ered using a cognitively motivated hierarchy of re- lations, and visual and discourse salience. 1 Introduction Our long-term goal is to develop conversational robots with whom we can interact through natural, fluent, visually situated dialog. An inherent as- pect of visually situated dialog is reference to ob- jects located in the physical environment (Moratz and Tenbrink, 2006). In this paper, we present a computational approach to the generation of spa- tial locative expressions in such situated contexts. The simplest form of locative expression is a prepositional phrase, modifying a noun phrase to locate an object. (1) illustrates the type of locative we focus on generating. In this paper we use the term target (T) to refer to the object that is being located by a spatial expression and the term land- mark (L) to refer to the object relative to which the target’s location is described. (1) a. the book [T] on the table [L] Generating locative expressions is part of the general field of generating referring expressions (GRE). Most GRE algorithms deal with the same problem: given a domain description and a target object, generate a description of the target object that distinguishes it from the other objects in the domain. We use distractor objects to indicate the ∗The research reported here was supported by the CoSy project, EU FP6 IST ”Cognitive Systems” FP6-004250-IP. objects in the context excluding the target that at a given point in processing fulfill the description of the target object that has been generated. The description generated is said to be distinguishing if the set of distractor objects is empty. Several GRE algorithms have addressed the is- sue of generating locative expressions (Dale and Haddock, 1991; Horacek, 1997; Gardent, 2002; Krahmer and Theune, 2002; Varges, 2004). How- ever, all these algorithms assume the GRE compo- nent has access to a predefined scene model. For a conversational robot operating in dynamic envi- ronments this assumption is unrealistic. If a robot wishes to generate a contextually appropriate ref- erence it cannot assume the availability of a fixed scene model, rather it must dynamically construct one. However, constructing a model containing all the relationships between all the entities in the do- main is prone to combinatorial explosion, both in terms of the number objects in the context (the lo- cation of each object in the scene …

Content truncated to 500 words