MovieChats: Chat like Humans in a Closed Domain

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6605–6619, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6605 MovieChats: Chat like Humans in a Closed Domain Hui Su1∗, Xiaoyu Shen2∗,Zhou Xiao1, Zheng Zhang3† Ernie Chang4, Cheng Zhang1, Cheng Niu1 and Jie Zhou1 1Pattern Recognition Center, Wechat AI, Tencent Inc, China 2Amazon Alexa AI 3Tsinghua University 4Department of Language Science and Technology, Saarland University aaronsu@tencent.com,gyouu@amazon.com Abstract Being able to perform in-depth chat with hu- mans in a closed domain is a precondition before an open-domain chatbot can ever be claimed. In this work, we take a close look at the movie domain and present a large-scale high-quality corpus with fine-grained annota- tions in hope of pushing the limit of movie- domain chatbots. We propose a unified, read- ily scalable neural approach which reconciles all subtasks like intent prediction and knowl- edge retrieval. The model is first pretrained on the huge general-domain data, then finetuned on our corpus. We show this simple neural approach trained on high-quality data is able to outperform commercial systems replying on complex rules. On both the static and interac- tive tests, we find responses generated by our system exhibits remarkably good engagement and sensibleness close to human-written ones. We further analyze the limits of our work and point out potential directions for future work 1. 1 Introduction Being able to converse like humans in a closed domain is a precondition before an intelligent open- domain chatbot, which further requires transiting among various domains, can be designed (Gao et al., 2019; Su et al., 2020). Nonetheless, even if constrained in a specific domain, current chatbots are still far from satisfactory. Unlike task-oriented systems that can be relatively well-resolved with handcrafted templates, human conversations fea- ture a complex mixture of QA, chitchat, recommen- dation, etc. without pre-specified goals or conver- sational patterns (Dodge et al., 2016; Akasaki and Kaji, 2017; Shen et al., 2018). Selecting proper ∗Corresponding Authors. Work done before Xiaoyu Shen joins Amazon. †Work done while interning at Wechat. 1Dataset and model are available at https://github. com/chin-gyou/MovieChats. domain knowledge to support response generation at all the different situations is challenging (Mil- ward and Beveridge, 2003; Lian et al., 2019; Shen et al., 2019). In this work, we direct our focus to the movie domain and present a large-scale, crowd- sourced Chinese dataset with fine-grained anno- tations in hope of boosting the study towards a human-like closed-domain chatbot. A variety of dialogue datasets with grounded do- main knowledge have already been proposed. How- ever, they are collected either through (1) online forum crawling (Dodge et al., 2016; Ghazvininejad et al., 2018; Liu et al., 2018; Zhou et al., 2018a; Qin et al., 2019), which are noisy, multi-party, mostly contain only single-exchange QA, or (2) crowd- sourced (Zhu et al., 2017; Zhou et al., 2018b; Moon et al., 2019; Wu et al., 2019), which are small-scale and often created in an overconstrained setting like teacher-student (Moghe et al., 2018). Even for datasets crowd-sourced in …

Content truncated to 500 words