Gamification Platform for Collecting Task-oriented Dialogue Data

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7084–7093 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7084 Gamification Platform for Collecting Task-oriented Dialogue Data Haruna Ogawa Hitoshi Nishikawa Takenobu Tokunaga Hikaru Yokono Tokyo Institute of Technology Fujitsu Laboratories Ltd. ogawa.h.ai@m.titech.ac.jp, {hitoshi, take}@c.titech.ac.jp yokono.hikaru@fujitsu.com Abstract Demand for massive language resources is increasing as the data-driven approach has established a leading position in natural language processing. However, creating dialogue corpora is still a difficult task due to the complexity of the human dialogue structure and the diversity of dialogue topics. Though crowdsourcing is majorly used to assemble such data, it presents problems such as less-motivated workers. We propose a platform for collecting task-oriented situated dialogue data by using gamification. Combining a video game with data collection benefits such as motivating workers and cost reduction. Our platform enables data collectors to create their original video game in which they can collect dialogue data of various types of tasks by using the logging function of the platform. Also, the platform provides the annotation function that enables players to annotate their own utterances. The annotation can be gamified as well. We aim at high-quality annotation by introducing such self-annotation method. We implemented a prototype of the proposed platform and conducted a preliminary evaluation to obtain promising results in terms of both dialogue data collection and self-annotation. Keywords: task-oriented dialogue, situated dialogue, data collection, gamification, dialogue act, annotation, Minecraft R⃝ 1. Introduction Demand for massive language resources is increasing as the data-driven approach using machine learning has es- tablished a leading position in natural language process- ing (NLP). The dialogue research is no exception to this tendency. Due to the complexity of the human dialogue structure and the diversity of dialogue topic, the data-driven approach for dialogue research requires a large amount of data (Ferreira et al., 2015) compared with other tasks like named entity recognition and syntactic analysis. For the ef- ficient collection of dialogue data in various domains, we propose a collection platform using gamification. Particu- larly we target collecting task-oriented dialogues. Dialogue can be classified into two types: non-task- oriented and task-oriented dialogues. The former has no specific goal to achieve through dialogue; it is also called chit-chat. Smooth continuation or just passing the time by chatting is the “goal” of this type of dialogue. In contrast, the task-oriented dialogue has a specific goal for dialogue participants to achieve together through dialogue. Recently, conversational assistants such as Apple’s Siri and Ama- zon’s Alexa are gaining popularity. In the dialogue with those systems, users usually have a specific goal, e.g. seek- ing certain information such as a nearby restaurant. Aside from the copyright issue, we could technically collect chit- chat data by crawling chit-chat logs on the Internet. We can also extract dialogue parts from existing texts, e.g. novels and movie scripts. On the other hand, the same methods are difficult to apply to the collection of task-oriented dia- logues. Since the goal …

Content truncated to 500 words