The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2453–2470 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2453 The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents Kurt Shuster, Da Ju, Stephen Roller Emily Dinan, Y-Lan Boureau, Jason Weston Facebook AI Research {kshuster,daju,roller,edinan,ylan,jase}@fb.com Abstract We introduce dodecaDialogue: a set of 12 tasks that measures if a conversational agent can communicate engagingly with personal- ity and empathy, ask questions, answer ques- tions by utilizing knowledge resources, dis- cuss topics and situations, and perceive and converse about images. By multi-tasking on such a broad large-scale set of data, we hope to both move towards and measure progress in producing a single unified agent that can perceive, reason and converse with humans in an open-domain setting. We show that such multi-tasking improves over a BERT pre- trained baseline, largely due to multi-tasking with very large dialogue datasets in a similar domain, and that the multi-tasking in general provides gains to both text and image-based tasks using several metrics in both the fine- tune and task transfer settings. We obtain state- of-the-art results on many of the tasks, provid- ing a strong baseline for this challenge. 1 Introduction One of the goals of AI is to build a seeing, talk- ing agent that can discuss, reason, empathize, and provide advice – in short a system that can per- form natural communication displaying many of the properties expected when speaking to a human partner. Ideally, it should be able to be knowl- edgeable and personable, expert and engaging, se- rious or humorous – depending on the situation. It should be capable of answering questions, asking questions, responding to statements, having its own persona, and grounding the dialogue with external information and images. While no single task exists that can train an agent or measure its ability on all of these axes at once, a number of distinct large-scale datasets targeting subsets of these skills have recently become avail- able. We thus assemble these disparate tasks to form a single challenge: dodecaDialogue, consist- ing of 12 subtasks. Each contains both training data to build the skills we desire for our agent, and validation and test sets to measure our agent’s abil- ity at that skill. The overall goal is a single agent that can display all these skills. As some of the subtasks have very large datasets, e.g. 2.2 billion utterances, they can possibly help the agent with other skills too. We thus build a model capable of training and multi-tasking on all these sources. We employ a transformer-based architecture (Vaswani et al., 2017) which accepts an image, external textual information and dialogue history as input, and gen- erates a response for a given dialogue turn. Practi- cally, by pre-training on the largest of the subtasks and then multi-tasking on all them, we can obtain state-of-the-art results compared to existing inde- pendently reported performance on all 10 of the 12 subtasks that have previous comparable results. We hence …

Content truncated to 500 words