Data Collection for Empirically Determining the Necessary Information for Smooth Handover in Dialogue

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4060–4068 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4060 Data Collection for Empirically Determining the Necessary Information for Smooth Handover in Dialogue Sanae Yamashita, Ryuichiro Higashinaka Graduate School of Informatics, Nagoya University Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan yamashita.sanae.w7@s.mail.nagoya-u.ac.jp, higashinaka@i.nagoya-u.ac.jp Abstract Despite recent advances, dialogue systems still struggle to achieve fully autonomous transactions. Therefore, when a system encounters a problem, human operators need to take over the dialogue to complete the transaction. However, it is unclear what information should be presented to the operator when this handover takes place. In this study, we conducted a data collection experiment in which one of two operators talked to a user and switched with the other operator periodically while exchanging notes when the handovers took place. By examining these notes, it is possible to identify the information necessary for handing over the dialogue. We collected 60 dialogues in which two operators switched periodically while performing chat, consultation, and sales tasks in dialogue. We found that adjacency pairs are a useful representation for recording conversation history. In ad- dition, we found that key-value-pair representation is also useful when there are underlying tasks, such as consultation and sales. Keywords: dialogue handover, discourse processing, adjacency pairs 1. Introduction In recent years, due to the advances in deep learning- based techniques, we are seeing the emergence of highly sophisticated dialogue systems (Brown et al., 2020; Pichl et al., 2020; Ou and Lin, 2020; Adiwardana et al., 2020; Roller et al., 2021). Despite this progress, dialogue systems still struggle to achieve fully au- tonomous transactions with users, necessitating human operators to take over the dialogue to complete the transactions when problems occur. However, it may be difficult for humans to take over the dialogue smoothly because the content of the dialogue may be complex and/or difficult to immediately grasp. Some systems show the raw dialogue history, highlighted keywords (e.g., Kawahara et al. (2021)), or summary (e.g., Ya- mashita and Higashinaka (2021)), but even so, it is not yet clear what kind of information is most effective for a smooth handover. Here, we define handover as join- ing a dialogue in the middle and achieving the original objective of the dialogue. In this study, in order to identify the necessary informa- tion for a smooth handover of dialogue, we conducted a data collection experiment in which one of two op- erators talked to a user and switched with the other operator periodically, exchanging notes when the han- dover took place (Fig. 1). By examining these notes, it may be possible to identify the information necessary for handing over the dialogue. In our experiment, we collected 60 dialogues in which two operators switched periodically while performing chat, consultation, and sales tasks in dialogue. Our analysis showed that adjacency pairs are a useful rep- resentation for recording conversation history. In addi- tion, we found that key-value-pair representation is also useful when there are …

Content truncated to 500 words