ELI5: Long Form Question Answering

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558–3567 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3558 ELI5: Long Form Question Answering Angela Fan1,2 Yacine Jernite⇤1 Ethan Perez⇤3 David Grangier4 Jason Weston1 Michael Auli1 1Facebook AI Research 2LORIA 3NYU ‡ 4Google AI ‡ [angelafan,yjernite,jase,michaelauli]@fb.com, perez@nyu.edu, grangier@google.com Abstract We introduce the first large-scale corpus for long-form question answering, a task requir- ing elaborate and in-depth answers to open- ended questions. The dataset comprises 270K threads from the Reddit forum “Explain Like I’m Five” (ELI5) where an online community provides answers to questions which are com- prehensible by five year olds. Compared to ex- isting datasets, ELI5 comprises diverse ques- tions requiring multi-sentence answers. We provide a large set of web documents to help answer the question. Automatic and human evaluations show that an abstractive model trained with a multi-task objective outper- forms conventional Seq2Seq, language mod- eling, as well as a strong extractive baseline. However, our best model is still far from hu- man performance since raters prefer gold re- sponses in over 86% of cases, leaving ample opportunity for future improvement.1 1 Introduction Existing question answering datasets have enabled significant progress in models that provide ex- tractive or unambigious short answers. However, less attention has been paid to open-ended ques- tions that require explanations. In this work, we present ELI5: a Long Form Question Answer- ing dataset that emphasizes the dual challenges of isolating relevant information within long source documents and generating paragraph-length ex- planations in response to complex, diverse ques- tions (see illustrations in Figures 1 and 2). The first challenge of ELI5 is the length and di- versity of answers that span multiple sentences: ⇤ Equal contribution ‡ Work done while at Facebook AI Research 1Dataset, Pretrained Models, and Additional Informa- tion is available: https://facebookresearch. github.io/ELI5, https://github.com/ facebookresearch/ELI5 Question: How do Jellyfish function without brains or ner- vous systems? [...] (60 words) Answer: Jellyfish may not have a brain, but they have a rough nervous system and innate behaviours. However, they are very simple creatures. They’re invertebrate: creatures with- out a backbone. Most jellyfish have really short life spans. Sometimes just a couple of hours. [...] As their name im- plies, they are largely composed of basically jelly inside a thin membrane. They’re over 95% water. (327 words) Documents: [...] Jellyfish do not have brains, and most barely have nervous systems. They have primitive nerve cells that help them orient themselves in the water and sense light and touch. [...] While they dont possess brains, the animals still have neurons that send all sorts of signals throughout their body. [...] They may accomplish this through the as- sistance of their nerve rings. Jellyfish don’t have brains, and that’s just where things begin. They don’t have many of the body parts that are typical in other animals. [...] (1070 words) Figure 1: ELI5 example. Models must write multi-sentence answers given questions and supporting web documents. questions are complex …

Content truncated to 500 words