Anaphora Resolution using Extended Centen’ng Algorithm in a Multi-modal Dialogue System

e e e O e O O e O O O O O O O O 0 O e O O O O O O O O O O e O O O e O O O O O Anaphora Resolution using an Extended Centering Algorithm in a Multi-modal Dialogue System Harksoo IGm, Jeong-Mi Cho, Jungyun SCO Department of Computer Science, Sogang University Seoul, 121-742, Korea hskim@nlpzodiac.sogang.ac.kr, jmcho@nlprep.sogang.ac.kr, seojy@ccs.sogang.ac:kr Abstract Anaphora in multi-modal dialogues have different aspects compared to the anaphora in language-only dialogues. They often refer to the items signified by a gesture or by visual means. In this paper, we define two kinds of anaphora: screen anaphora and referring anaphora, and propose two general methods to resolve these anaphora. One is a simple mapping algorithm that can find items refetxed with/without pointing gestures on a screen. The other is the centering algorithm with a dual cache model, which Walker's centering algorithm is extended to for a multi-modal dialogue system. The extended algorithm is appropriate to resolve various anaphora in a multi-modal dialogue .because it keeps utterances, visual information and screen switching- ,.tim.e. In the experiments, the system. Correctly resolved 384 anaphora out of 402 anaphom in 40 dialogues (0..54 anaphom per utterance) showing 95.5% correctness. Introduction Human face<o-face communication is an ideal model for humm-computa- interface. One of the major features of face-to-face communication is its multiplicity of communication channels that acts on multiple modalities. By providing a number of channels through which information may pass between a user and a computer, a multi-modal dialogue system gives the user a • more convenient and natural interface than a language-only dialogue system. In the system, a user often uses a variety of anaphodc expressions like this, the red item, it, etc. User's intention is passed to the system through multiple channels, e.g., the auditory channel (carrying speech) and the visual channel (c~nrying gestures and/or facial expressions). For example, a user can say utterance (4) in Figure ! ! while touching an item on the screen. The user may also say utterance (8) without touching the screen when there is only one red item displayed on the screen. Moreover, the user can use anaphoric expression to refer to an entity in previous utterances as in utterance (10). (1) S: May I help you7 (2) U: I want to see some desks. (3) S: (displaying mode/200 and mode/250) We have these modeb. (4) U: (pointing to the model200) How much is thLv? (5) S: It is 150,000 Won. (6) U: I'd like to see some chairs, too. (7) S: (displaying model 100 and model 150) We have t/w~re medeb. (8) U: How much is the red item? (9) S: It is 80,000 Won. (10) U: (pointing to the model 100) l'd like to buy thb and tke prev/ous se/ect/on. Figure 1: Motivational example I Previous rescaw, h on a multi-modal dialogue system was focused on finding the relationship between a pointing gesture and a deictic expression (Bolt (1980), Neal et al. …

Content truncated to 500 words