Towards a Model of Face-to-Face Grounding

Towards a Model of Face-to-Face Grounding Yukiko I. Nakano†/†† Gabe Reinstein† Tom Stocky† Justine Cassell† †MIT Media Laboratory E15-315 20 Ames Street Cambridge, MA 02139 USA {yukiko, gabe, tstocky, justine}@media.mit.edu ††Research Institute of Science and Technology for Society (RISTEX) 2-5-1 Atago Minato-ku, Tokyo 105-6218, Japan nakano@kc.t.u-tokyo.ac.jp Abstract We investigate the verbal and nonverbal means for grounding, and propose a design for embodied conversational agents that re- lies on both kinds of signals to establish common ground in human-computer inter- action. We analyzed eye gaze, head nods and attentional focus in the context of a di- rection-giving task. The distribution of nonverbal behaviors differed depending on the type of dialogue move being grounded, and the overall pattern reflected a monitor- ing of lack of negative feedback. Based on these results, we present an ECA that uses verbal and nonverbal grounding acts to up- date dialogue state. 1 Introduction An essential part of conversation is to ensure that the other participants share an understanding of what has been said, and what is meant. The proc- ess of ensuring that understanding – adding what has been said to the common ground – is called grounding [1]. In face-to-face interaction, nonver- bal signals as well as verbal participate in the grounding process, to indicate that an utterance is grounded, or that further work is needed to ground. Figure 1 shows an example of human face-to-face conversation. Even though no verbal feedback is provided, the speaker (S) continues to add to the directions. Intriguingly, the listener gives no ex- plicit nonverbal feedback – no nods or gaze to- wards S. S, however, is clearly monitoring the listener’s behavior, as we see by the fact that S looks at her twice (continuous lines above the words). In fact, our analyses show that maintaining focus of attention on the task (dash-dot lines un- derneath the words) is the listener’s public signal of understanding S’s utterance sufficiently for the task at hand. Because S is manifestly attending to this signal, the signal allows the two jointly to rec- ognize S’s contribution as grounded. This paper provides empirical support for an essential role for nonverbal behaviors in grounding, motivating an architecture for an embodied conversational agent that can establish common ground using eye gaze, head nods, and attentional focus. Although grounding has received significant at- tention in the literature, previous work has not ad- dressed the following questions: (1) what predictive factors account for how people use non- verbal signals to ground information, (2) how can a model of the face-to-face grounding process be used to adapt dialogue management to face-to-face conversation with an embodied conversational agent. This paper addresses these issues, with the goal of contributing to the literature on discourse phenomena, and of building more advanced con- versational humanoids that can engage in human conversational protocols. In the next section, we discuss relevant previous work, report results from our own empirical study and, based on our analysis of conversational data, propose a model of grounding using both …

Content truncated to 500 words