Incrementally Tracking Reference in Human/Human Dialogue Using Linguistic and Extra-Linguistic Information

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 272–282, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Incrementally Tracking Reference in Human/Human Dialogue Using Linguistic and Extra-Linguistic Information Casey Kennington CITEC, Bielefeld University Universit¨atsstraße 25 Bielefeld Germany ckennington@cit- ec.uni-bielefeld.de Ryu Iida NICT UCRI Seika-cho, Soraku-gun Kyoto, Japan ryu.iida@nict.go.jp Takenobu Tokunaga Tokyo Institute of Technology ˆOokayama, Meguro Tokyo, Japan take@cs.titech.ac.jp David Schlangen Bielefeld University Universit¨atsstraße 25 Bielefeld Germany david.schlangen@ uni-bielefeld.de Abstract A large part of human communication in- volves referring to entities in the world and often these entities are objects that are visu- ally present for the interlocutors. A system that aims to resolve such references needs to tackle a complex task: objects and their vi- sual features need to be determined, the re- ferring expressions must be recognised, and extra-linguistic information such as eye gaze or pointing gestures need to be incorporated. Systems that can make use of such informa- tion sources exist, but have so far only been tested under very constrained settings, such as WOz interactions. In this paper, we apply to a more complex domain a reference resolution model that works incrementally (i.e., word by word), grounds words with visually present properties of objects (such as shape and size), and can incorporate extra-linguistic informa- tion. We find that the model works well com- pared to previous work on the same data, de- spite using fewer features. We conclude that the model shows potential for use in a real- time interactive dialogue system. 1 Introduction Referring to entities in the world via definite de- scriptions makes up a large part of human commu- nication (Poesio and Vieira, 1997). In task-oriented situations, these references are often to entities that are visible in the shared environment. This kind of reference has attracted attention in recent computa- tional research, but the kinds of interactions stud- ied are often fairly restricted in controlled lab situ- ations (Tanenhaus and Spivey-Knowlton, 1995) or simulated human/computer interactions, (Schlangen et al., 2009; Kousidis et al., 2013; Chai et al., 2014). In such task-oriented, co-located settings, interlocu- tors can make use of extra-linguistic cues such as gaze or pointing gestures. Furthermore, listeners resolve references as they unfold, often identifying the referred entity before the end of the reference (Tanenhaus and Spivey-Knowlton, 1995; Spivey et al., 2002), however research in reference resolution has mostly focused on full, completed referring ex- pressions. In this paper we make a first move towards ad- dressing somewhat more complex domains. We ap- ply a model of reference resolution, which has been tested in a simpler setup, on more natural data com- ing from a corpus of human/human interactions. The model is incremental in that it does not wait un- til the end of an utterance to process, rather it up- dates its interpretation at each word increment. The model can also incorporate other modalities, such as gaze or pointing cues (deixis) incrementally. We also model the saliency of …

Content truncated to 500 words