Generating Responses that Reflect Meta Information in User-Generated Question Answer Pairs

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5433–5441 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5433 Generating Responses that Reflect Meta Information in User-Generated Question Answer Pairs Takashi Kodama1∗, Ryuichiro Higashinaka2, Koh Mitsuda2, Ryo Masumura2, Yushi Aono2, Ryuta Nakamura3, Noritake Adachi3 and Hidetoshi Kawabata3 1Kyoto University, Japan 2NTT Corporation, Japan 3DWANGO Co., Ltd, Japan kodama@nlp.ist.i.kyoto-u.ac.jp {ryuichiro.higashinaka.tp, koh.mitsuda.td, ryou.masumura.ba, yushi.aono.dy}@hco.ntt.co.jp {ryuuta_nakamura, noritake_adachi, hidetoshi_kawabata}@dwango.co.jp Abstract This paper concerns the problem of realizing consistent personalities in neural conversational modeling by using user generated question-answer pairs as training data. Using the framework of role play-based question-answering, we collected single-turn question-answer pairs for particular characters from online users. Meta information was also collected such as emotion and intimacy related to question-answer pairs. We verified the quality of the collected data and, by subjective evaluation, we also verified their usefulness in training neural conversational models for generating utterances reflecting the meta information, especially emotion. Keywords: data collection, meta information, user-generated content, utterance generation, question-answering 1. Introduction Much attention has been paid to non-task oriented dialogue systems from their social and entertainment aspects (Wal- lace, 2009; Banchs and Li, 2012; Higashinaka et al., 2014). To make such systems more engaging, it is necessary that they exhibit consistent personalities. In neural conversa- tional modeling (Sordoni et al., 2015; Vinyals and Le, 2015; Shang et al., 2015; Serban et al., 2016), various architec- tures have been proposed to realize consistent personalities, such as incorporating user IDs (Li et al., 2016), personal attributes (Qian et al., 2018), and profile text (Zhang et al., 2018). However, the problem is that it is still costly to collect dialogue data for training such neural models. For efficiently collecting dialogue data (we deal with sin- gle question-answer pairs as dialogue data in this paper) for a particular character, a data collection method called “role play-based question-answering” has been proposed (Hi- gashinaka et al., 2018) (See Section 2. for details). In this method, fans of a particular character voluntarily provide question-answer pairs by playing the role of that character. It has been demonstrated that high-quality question-answer pairs can be efficiently collected and that dialogue systems that exhibit consistent personalities can be realized (Hi- gashinaka et al., 2018). This paper extends this work and aims to collect question- answer pairs for particular characters together with other pieces of information (called “meta information”), such as emotion and intimacy levels. The aim of collecting this additional data is to realize dialogue systems whose utterances can be controlled to reflect such meta information (Zhou et al., 2018; Zhou and Wang, 2018; Song et al., 2019). This is a useful feature when we want to realize systems that are affective and can become more intimate as an interaction progresses (Zhou and Wang, 2018). We verify ∗Work carried out during an internship at NTT Corporation. the quality of the collected data and empirically show that conversational models that exhibit consistent personalities as well as meta information, especially emotion, can …

Content truncated to 500 words