Towards Situated Dialogue: Revisiting Referring Expression Generation

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 392–402, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Towards Situated Dialogue: Revisiting Referring Expression Generation Rui Fang, Changsong Liu, Lanbo She, Joyce Y. Chai Department of Computer Science and Engineering Michigan State University, East Lansing, MI, 48824, USA {fangrui,cliu,shelanbo,jchai}@cse.msu.edu Abstract In situated dialogue, humans and agents have mismatched capabilities of perceiving the shared environment. Their representations of the shared world are misaligned. Thus referring expression generation (REG) will need to take this discrepancy into consider- ation. To address this issue, we developed a hypergraph-based approach to account for group-based spatial relations and uncertain- ties in perceiving the environment. Our em- pirical results have shown that this approach outperforms a previous graph-based approach with an absolute gain of 9%. However, while these graph-based approaches perform effec- tively when the agent has perfect knowledge or perception of the environment (e.g., 84%), they perform rather poorly when the agent has imperfect perception of the environment (e.g., 45%). This big performance gap calls for new solutions to REG that can mediate a shared perceptual basis in situated dialogue. 1 Introduction Situated human robot dialogue has received increas- ing attention in recent years. In situated dialogue, robots/artificial agents and their human partners are co-present in a shared physical world. Robots need to automatically perceive and make inference of the shared environment. Due to its limited perceptual and reasoning capabilities, the robot’s representation of the shared world is often incomplete, error-prone, and significantly mismatched from that of its human partner’s. Although physically co-present, a joint perceptual basis between the human and the robot cannot be established (Clark and Brennan, 1991). Thus, referential communication between the hu- man and the robot becomes difficult. How this mismatched perceptual basis affects ref- erential communication in situated dialogue was in- vestigated in our previous work (Liu et al., 2012). In that work, the main focus is on reference resolu- tion: given referential descriptions from human part- ners, how to identify referents in the environment even though the robot only has imperfect percep- tion of the environment. Since robots need to col- laborate with human partners to establish a joint per- ceptual basis, referring expression generation (REG) becomes an equally important problem in situated dialogue. Robots have much lower perceptual capa- bilities of the environment than humans. How can a robot effectively generate referential descriptions about the environment so that its human partner can understand which objects are being referred to? There has been a tremendous amount of work on referring expression generation in the last two decades (Dale, 1995; Krahmer and Deemter, 2012). However, most existing REG algorithms were devel- oped and evaluated under the assumption that agents and humans have access to the same kind of domain information. For example, many experimental se- tups (Gatt et al., 2007; Viethen and Dale, 2008; Golland et al., 2010; Striegnitz et al., 2012) were developed based on a visual world for which the in- ternal …

Content truncated to 500 words