Predicting the Resolution of Referring Expressions from User Behavior

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1354–1359, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Predicting the resolution of referring expressions from user behavior Nikos Engonopoulos1 Mart´ın Villalba1 Ivan Titov2 Alexander Koller1 1University of Potsdam, Germany 2University of Amsterdam, Netherlands {nikolaos.engonopoulos, martin.villalba}@uni-potsdam.de titov@uva.nl, koller@ling.uni-potsdam.de Abstract We present a statistical model for predicting how the user of an interactive, situated NLP system resolved a referring expression. The model makes an initial prediction based on the meaning of the utterance, and revises it con- tinuously based on the user’s behavior. The combined model outperforms its components in predicting reference resolution and when to give feedback. 1 Introduction Speakers and listeners in natural communication are engaged in a highly interactive process. In order to achieve some communicative goal, the speaker will perform an utterance which they believe has a high chance of achieving that goal. They will then moni- tor the listener’s behavior to see whether this goal is actually being achieved. This process is a core part of what is commonly called grounding in the dia- logue literature (see e.g. (Clark, 1996; Traum, 1994; Paek and Horvitz, 1999; Hirst et al., 1994)). Inter- active computer systems that are to carry out an ef- fective and efficient conversation with a user must model this grounding process, and should ideally re- spond to the user’s observed behavior in real time. For instance, if the user of a pedestrian navigation system takes a wrong turn, the system should inter- pret this as evidence of misunderstanding and bring the user back on track. We focus here on the problem of predicting how the user has resolved a referring expression (RE) that was generated by the system, i.e. a noun phrase that is intended to identify some object uniquely to the listener. A number of authors have recently offered statistical models for parts of this problem. Golland et al. (2010) and Garoufi and Koller (2011) have presented log-linear models for predicting how the listener will resolve a given RE in a given scene; however, these models do not update the probabil- ity model based on observing the user’s reactions. Nakano et al. (2007), Buschmeier and Kopp (2012), and Koller et al. (2012) all predict what the listener understood based on their behavior, but do not con- sider the RE itself in the model. The models of Frank and Goodman (2012) and Vogel et al. (2013) aim at explaining the effect of implicatures on the listener’s RE resolution process in terms of hypothe- sized interactions, but do not actually support a real- time interaction between a system and a user. In this paper, we show how to predict how the listener has resolved an RE by combining a statis- tical model of RE resolution based on the RE itself with a statistical model of RE resolution based on the listener’s behavior. To our knowledge, this is the first approach to combine two such models ex- plicitly. We consider …

Content truncated to 500 words