LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15387–15405 July 9-14, 2023 ©2023 Association for Computational Linguistics LiveChat: A Large-Scale Personalized Dialogue Dataset Automatically Constructed from Live Streaming Jingsheng Gao1,2∗, Yixin Lian2, Ziyi Zhou2, Yuzhuo Fu1†, Baoyuan Wang2† 1 School of SEIEE, Shanghai Jiao Tong University, China 2 Xiaobing.AI {gaojingsheng, yzfu}@sjtu.edu.cn {lianyixin, zhouziyi, wangbaoyuan}@xiaobing.ai Abstract Open-domain dialogue systems have made promising progress in recent years. While the state-of-the-art dialogue agents are built upon large-scale text-based social media data and large pre-trained models, there is no guaran- tee these agents could also perform well in fast-growing scenarios, such as live stream- ing, due to the bounded transferability of pre- trained models and biased distributions of pub- lic datasets from Reddit and Weibo, etc. To improve the essential capability of responding and establish a benchmark in the live open- domain scenario, we introduce the LiveChat dataset, composed of 1.33 million real-life Chi- nese dialogues with almost 3800 average ses- sions across 351 personas and fine-grained pro- files for each persona. LiveChat is automati- cally constructed by processing numerous live videos on the Internet and naturally falls within the scope of multi-party conversations, where the issues of Who says What to Whom should be considered. Therefore, we target two criti- cal tasks of response modeling and addressee recognition and propose retrieval-based base- lines grounded on advanced techniques. Ex- perimental results have validated the positive effects of leveraging persona profiles and larger average sessions per persona. In addition, we also benchmark the transferability of advanced generation-based models on LiveChat and pose some future directions for current challenges. 1 1 Introduction Building dialogue systems to converse naturally with humans has been one of the longest-running goals in artificial intelligence (Zhou et al.; Roller et al., 2021). To usher that chatbot response prop- erly in diverse scenarios, it is desirable to train a conversational agent based on massive large-scale ∗Work done during an internship at Xiaobing.AI †Corresponding Author 1The code and dataset will be publicly available at https://github.com/gaojingsheng/LiveChat. 对，每天10点半开始。 Yes, I start at 10:30 everyday. 54321 6 听说潮汕美食很多。 It is heard that Chaoshan has a lot of delicious food. 我投上海一票。 I vote for Shanghai. 惨了，坠入爱河了。 Oh no I am falling into love. 每天都直播吗？ Are you streaming everyday? 我连续看三天了。 I have been watching for three days. Response-to 自从看你直播之后已经看不了别的主播了。 I haven’t watched anyone else’s since I watched your live streaming. Figure 1: A session example of LiveChat. A streamer will respond to one audience’s comment from the com- ments area. datasets with multiple domains. Current dialogue datasets mainly leverage online forum posts to build reply-to relationships between users, such as Reddit (Mazaré et al., 2018; Zhong et al., 2020) and Weibo (Zheng et al., 2019; Qian et al., 2021). Despite the scalability and diversity of current di- alogue corpora, dialogue models pre-trained on these conversation datasets can not perform effec- tively when applied to a completely new domain, such as live streaming. The reason lies in the intrin- sic …

Content truncated to 500 words