Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2128 - 2150 July 10-15, 2022 ©2022 Association for Computational Linguistics Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models Sanghwan Bae1 Donghyun Kwak1 Sungdong Kim2 Donghoon Ham1 Soyoung Kang1 Sang-Woo Lee1,2 Woomyoung Park1 NAVER CLOVA1 NAVER AI Lab2 {sanghwan.bae, donghyun.kwak, sungdong.kim, donghoon.ham, sy.kang, sang.woo.lee, max.park}@navercorp.com Abstract Recent open-domain dialogue models have brought numerous breakthroughs. However, building a chat system is not scalable since it often requires a considerable volume of human-human dialogue data, especially when enforcing features such as persona, style, or safety. In this work, we study the challenge of imposing roles on open-domain dialogue systems, with the goal of making the sys- tems maintain consistent roles while convers- ing naturally with humans. To accomplish this, the system must satisfy a role speci- fication that includes certain conditions on the stated features as well as a system pol- icy on whether or not certain types of utter- ances are allowed. For this, we propose an efficient data collection framework leveraging in-context few-shot learning of large-scale lan- guage models for building role-satisfying dia- logue dataset from scratch. We then compare various architectures for open-domain dia- logue systems in terms of meeting role specifi- cations while maintaining conversational abil- ities. Automatic and human evaluations show that our models return few out-of-bounds ut- terances, keeping competitive performance on general metrics. We release a Korean dialogue dataset we built for further research1. 1 Introduction Recent large-scale language models (LMs) have brought numerous breakthroughs in open-domain dialogue systems, yielding human-like responses (Zhang et al., 2020; Adiwardana et al., 2020; Brown et al., 2020; Roller et al., 2021; Kim et al., 2021a). In addition, there have been progresses in controlling dialogue systems in persona, style, and safety (Zhang et al., 2018; Smith et al., 2020; Xu et al., 2021), which impose consistency on chat- bot’s personality and mitigate undesirable features 1The dataset is available at https://github.com/ naver-ai/carecall-corpus Figure 1: An example of a chatbot system that cares for senior citizens living alone. The utterance in red highlights the model’s mistaken identity as a chef rather than the caring chatbot. such as toxic or biased language. However, build- ing a chatbot system combining these capabilities is still challenging, which requires numerous human- human dialogues for those conversational skills. Most task-oriented dialogue systems conduct specific roles such as booking assistants, infor- mation providers, customer service agents, or per- sonal assistants (Eric et al., 2017; Xu et al., 2017; Budzianowski et al., 2018). However, studies on open-domain dialogue systems that perform spe- cific roles have been insufficiently investigated, even though the role can be defined for the practical chatbot systems (e.g., chatbots that care for senior citizens living alone, or counseling chatbots). In these cases, the chatbot systems do not have an ex- plicit goal or task other than to proactively engage in conversations, but may have system policies on whether or not certain types …

Content truncated to 500 words