Predicting Nods by using Dialogue Acts in Dialogue

Predicting Nods by using Dialogue Acts in Dialogue Ryo Ishii, Ryuichiro Higashinaka, Junji Tomita NTT Media Intelligence Laboratories, NTT Corporation 1-1 Hikari-no-oka, Yokosuka-shi, Kanagawa, Japan {ishii.ryo, higashinaka.ryuichiro, tomita.junji@lab.ntt.co.jp Abstract In addition to verbal behavior, nonverbal behavior is an important aspect for an embodied dialogue system to be able to conduct a smooth conversation with the user. Researchers have focused on automatically generating nonverbal behavior from speech and language information of dialogue systems. We propose a model to generate head nods accompanying utterance from natural language. To the best of our knowledge, previous studies generated nods from the final morphemes at the end of an utterance. In this study, we focused on dialog act information indicating the intention of an utterance and determined whether this information is effective for generating nods. First, we compiled a Japanese corpus of 24 dialogues including utterance and nod information. Next, using the corpus, we created a model that estimates whether a nod occurs during an utterance by using a morpheme at the end of a speech and dialog act. The results show that our estimation model incorporating dialog acts outperformed a model using morpheme information. The results suggest that dialog acts have the potential to be a strong predictor with which to generate nods automatically. Keywords: nod, dialogue act, japanese dialogue 1. Introduction Nonverbal behavior in human communication has impor- tant functions of transmitting emotions and intentions in ad- dition to verbal behavior (BirdWhistell, 1970). This means that an embodied dialogue system should be able to ex- press nonverbal behavior according to the utterance to com- municate smoothly with the user (McBreen and Jack, 2001; Watanabe et al., 2003; Ishi et al., 2010). Against such a background, researchers have focused on construct- ing automatic generation models of nonverbal behav- ior from speech and linguistic information. Among nonverbal behaviors, nodding of the head is very important for emphasizing speech, giving and re- ceiving speech authority, giving feedback, expressing conversational engagement, and intention of starting to speak (Senko Maynard, 1987; Senko Maynard, 1989; Ishii et al., 2015b; Ishii et al., 2017b; Ooko et al., 2011). It has been shown that nodding improves the naturalness of avatars and dialog systems and promotes conversation. Nodding accompanying an utterance has the effect of strengthening the persuasive power of speech and mak- ing it easier for the conversational partner to understand the contents of the utterance (Lohse et al., 2014). Re- searchers have tried to generate nods during speaking from speech and natural language. In particular, they used sev- eral acoustic features, such as sound pressure and prosody, for generating nods (Yehia et al., 2002; KG et al., 2004; Graf et al., 2002; Busso et al., 2007; Beskow et al., 2006; Iwano et al., 1996; Ishi et al., 2010). However, it has been difficult to accurately generate nods at an appropriate time according to an utterance from only speech. A few studies have tackled the problem of generating nods from natural language. These studies focused on the final morphemes in the …

Content truncated to 500 words