Automatic Generation of Large-scale Multi-turn Dialogues from Reddit

Proceedings of the 29th International Conference on Computational Linguistics, pages 3360–3373 October 12–17, 2022. 3360 Generation of Large-scale Multi-turn Dialogues from Reddit Daniil Huryn Computer Science Emory University Atlanta, GA, USA danikhur@gmail.com William M. Hutsell Computer Science Emory University Atlanta, GA, USA mack.hutsell@gmail.com Jinho D. Choi Computer Science Emory University Atlanta, GA, USA jinho.choi@emory.edu Abstract This paper presents novel methods to automat- ically convert posts and their comments from discussion forums such as Reddit into multi- turn dialogues. Our methods are generalizable to any forums; thus, they allow us to generate a massive amount of dialogues for diverse topics that can be used to pretrain language models. Four methods are introduced, GreedyBaseline, GreedyAdvanced, Beam Search and Threading, which are applied to posts from 10 subreddits and assessed. Each method makes a noticeable improvement over its predecessor such that the best method shows an improvement of 36.3% over the baseline for appropriateness. Our best method is applied to posts from those 10 sub- reddits for the creation of a corpus comprising 10,098 dialogues (3.3M tokens), 570 of which are compared against dialogues in three other datasets, Blended Skill Talk, Daily Dialogue, and Topical Chat. Our dialogues are found to be more engaging but slightly less natural than the ones in the other datasets, while it costs a fraction of human labor and money to generate our corpus compared to the others. To the best of our knowledge, it is the first work to create a large multi-turn dialogue corpus from Reddit that can advance neural dialogue systems. 1 Introduction With the advent of encoder-decoder frameworks (Brown et al., 2020; Lewis et al., 2020; Raffel et al., 2020), neural-based open-domain dialogue models have recently gained a tremendous interest as they start sounding more human-like than ever (Adiwardana et al., 2020; Zhang et al., 2020; Roller et al., 2021). Training robust neural-based models requires a huge amount of dialogue data in numerous topics that is difficult to procure as real human-to-human conversations are expensive and time-intensive to conduct (Godfrey et al., 1992). Several studies have presented large dialogue data created by crowdsourcing (Zhang et al., 2018; Di- nan et al., 2019; Rashkin et al., 2019; Gopalakr- ishnan et al., 2019). However, it still requires non- trivial configurations in the cloud platform and the performance of crowd workers needs to be moni- tored constantly while paying them and the service a good amount of fees. Most encoder-decoder models used in dialogue systems are not pretrained on dialogues, just fine- tuned on relatively small dialogue datasets, which is a limiting factor. Few studies have utilized com- ment threads in discussion forums for the creation of dialogue data and enhanced the performance of dialogue systems (Al-Rfou et al., 2016; Mazaré et al., 2018). However, these comment-originated dialogues tend to be short and not as sensible due to a lack of contexts from the main posts that are unsuitable for training multi-turn dialogue models. Such data scarcity points toward a necessity for a parameterized model that generates …

Content truncated to 500 words