Complex Reading Comprehension Through Question Decomposition

Complex Reading Comprehension Through Question Decomposition Xiao-Yu Guo , Yuan-Fang Li , and Gholamreza Haffari Faculty of Information Technology, Monash University, Melbourne, Australia {xiaoyu.guo,yuanfang.li,gholamreza.haffari}@monash.edu Abstract Multi-hop reading comprehension requires not only the ability to reason over raw text but also the ability to combine multiple evidence. We propose a novel learning approach that helps language models better understand dif- ficult multi-hop questions and perform “com- plex, compositional” reasoning. Our model first learns to decompose each multi-hop ques- tion into several sub-questions by a train- able question decomposer. Instead of answer- ing these sub-questions, we directly concate- nate them with the original question and con- text, and leverage a reading comprehension model to predict the answer in a sequence- to-sequence manner. By using the same lan- guage model for these two components, our best seperate/unified t5-base variants outper- form the baseline by 7.2/6.1 absolute F1 points on a hard subset of DROP dataset. 1 Introduction Multi-hop Reading Comprehension (RC) is a chal- lenging problem that requires compositional, sym- bolic and arithmetic reasoning capabilities. Facing a difficult question, humans tend to first decom- pose it into several sub-questions whose answers can be more easily identified. The final answer to the overall question can then be concluded from the aggregation of all sub-questions’ answers. For instance, for the question in Table 1, we can natu- rally decompose it into three simpler sub-questions (1) “return the touchdown yards”, (2) “return the fewest of #1”, and (3) “return who caught #2”. The tokens #1 and #2 are the answers to the first and second sub-questions respectively. Finally, the player with the touchdown of #2 is returned as the final answer. State-of-the-art RC techniques employ large- scale pre-trained language models (LMs) such as GPT-3 (Brown et al., 2020) for their superior rep- resentation and reasoning capablities. Chain of C First, Detroit’s Calvin Johnson caught a 1-yard pass in the third quarter. The game’s final points came when Mike Williams of Tampa Bay caught a 5-yard. Q Who caught the touchdown for the fewest yards? Q1 return the touchdown yards Q2 return the fewest of #1 Q3 return who caught #2 A Calvin Johnson Table 1: An example for reading comprehension. C is the context, Q is a hard multi-hop question, and Q1, Q2, Q3 are sub-questions annotated in BREAK dataset. A is the answer to Q. thought prompting (Wei et al., 2022) elicits strong reasoning capability of LMs by providing inter- mediate reasoning steps. Least-to-most prompt- ing (Zhou et al., 2022) further shows the feasibility of conducting decomposition and multi-hop rea- soning, which happen on the decoder side together with the answer prediction procedure. However, compared to supervised learning models, both of these methods rely on extremely large LMs with tens and hundreds of billions of parameters to achieve competitive performance, thus requiring expensive hardware and incurring a large computa- tion footprint. Despite significant research on RC (Dua et al., 2019; Perez et al., 2020), those questions that re- quire strong compositional generalisability and nu- merical …

Content truncated to 500 words