Self-Contained Utterance Description Corpus for Japanese Dialog

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 1249–1255 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 1249 Self-Contained Utterance Description Corpus for Japanese Dialog Yuta Hayashibe Megagon Labs, Tokyo, Japan, Recruit Co., Ltd. 7-3-5 Ginza Chuo-ku, Tokyo, 104-8227, Japan hayashibe@megagon.ai Abstract Often both an utterance and its context must be read to understand its intent in a dialog. Herein we propose a task, Self- Contained Utterance Description (SCUD), to describe the intent of an utterance in a dialog with multiple simple natural sen- tences without the context. If a task can be performed concurrently with high accuracy as the conversation continues such as in an accommodation search dialog, the operator can easily suggest candidates to the customer by inputting SCUDs of the customer’s utterances to the accommodation search system. SCUDs can also describe the transition of customer requests from the dialog log. We construct a Japanese corpus to train and evaluate automatic SCUD generation. The corpus consists of 210 dialogs containing 10,814 sentences. We conduct an experiment to verify that SCUDs can be automatically generated. Additionally, we investigate the influence of the amount of training data on the automatic generation performance using 8,200 additional examples. Keywords: Dialog corpus, Natural language understanding, Utterance description, Text generation 1. Introduction To develop a task-oriented dialog system that responds appropriately to input utterances, the intent of the ut- terances must be correctly understood. For example, if a customer says, “Looks good, but expensive?”, the omitted phrases and the intent of the question must be recognized. Such an understanding of language has been formulated and studied using two major task frameworks. One is dialog-act classification or slot filling (Liu and Lane, 2016; Gupta et al., 2019; Shi, 2020). For exam- ple, MultiWOZ (Budzianowski et al., 2018) defines act types and slots for task-oriented dialogs in seven do- mains such as hotels and restaurants. They require pre- defined labels or slots. This task framework is powerful if the dialog proceeds according to pre-defined scenar- ios. However, it lacks flexibility because it can only interpret utterances using pre-defined types and slots. Therefore, interpreting utterances to solve tasks in an exploratory way in consultative dialogs is difficult us- ing this framework. The other framework is the generation of summary text. This can handle a wide variety of utterances. Recently, studies have employed this framework in the medical domain (Joshi et al., 2020; Song et al., 2020; Krishna et al., 2021) or the call center domain (Favre et al., 2015). In these studies, the whole dialog is interpreted rather than the intentions of individual utterances. This ap- proach can understand dialog after it is over, but is not suitable to comprehend dialog said in real time. Therefore, we propose a task, Self-Contained Utter- ance Description (SCUD), to simultaneously describe the intent of an utterance in a dialog with multiple simple natural sentences. By automatically generating such sentences, a dialog system can be connected to an information retrieval system …

Content truncated to 500 words