Unsupervised Question Decomposition for Question Answering

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8864–8880, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8864 Unsupervised Question Decomposition for Question Answering Ethan Perez1 2 Patrick Lewis1 3 Wen-tau Yih1 Kyunghyun Cho2 4∗ Douwe Kiela1 1Facebook AI Research, 2New York University, 3University College London, 4CIFAR Azrieli Global Scholar perez@nyu.edu Abstract We aim to improve question answering (QA) by decomposing hard questions into simpler sub-questions that existing QA systems are capable of answering. Since labeling ques- tions with decompositions is cumbersome, we take an unsupervised approach to pro- duce sub-questions, also enabling us to lever- age millions of questions from the internet. Specifically, we propose an algorithm for One-to-N Unsupervised Sequence transduc- tion (ONUS) that learns to map one hard, multi-hop question to many simpler, single- hop sub-questions. We answer sub-questions with an off-the-shelf QA model and give the resulting answers to a recomposition model that combines them into a final answer. We show large QA improvements on HOTPOTQA over a strong baseline on the original, out-of- domain, and multi-hop dev sets. ONUS auto- matically learns to decompose different kinds of questions, while matching the utility of su- pervised and heuristic decomposition methods for QA and exceeding those methods in flu- ency. Qualitatively, we find that using sub- questions is promising for shedding light on why a QA system makes a prediction.1 1 Introduction It has been a long-standing challenge in AI to an- swer questions of any level of difficulty (Winograd, 1991). Question answering (QA) systems strug- gle to answer complex questions such as “What profession do H. L. Mencken and Albert Camus have in common?” since the required informa- tion is scattered in different places (Yang et al., 2018). However, QA systems accurately answer ∗KC was a part-time research scientist at Facebook AI Research while working on this paper. 1Our code, data, and pretrained models are avail- able at https://github.com/facebookresearch/ UnsupervisedDecomposition. What profession do H. L. Mencken and Albert Camus have in common? Unsupervised Decomp. Model (ONUS) What profession does H. L. Mencken have? Single Hop QA Model Single Hop QA Model Henry Louis Mencken (1880 – 1956) was an American journalist, critic and scholar of American English. Albert Camus (7 November 1913 – 4 January 1960) was a French philosopher, author, and journalist. journalist Q SQ1 SQ2 A1 A2 A Who was Albert Camus? Passage Recomposition Model Figure 1: Overview: Using unsupervised learning, we decompose a multi-hop question into single-hop sub- questions, whose predicted answers are given to a re- composition model to predict the final answer. simpler, related questions such as “What profes- sion does H. L. Mencken have?” and “Who was Al- bert Camus?” (Petrochuk and Zettlemoyer, 2018). Thus, a promising strategy to answer hard ques- tions is divide-and-conquer: decompose a hard question into simpler sub-questions, answer the sub-questions with a QA system, and recompose the resulting answers into a final answer, as shown in Figure 1. This approach leverages strong perfor- mance on simple questions to help answer …

Content truncated to 500 words