Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6064–6080 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6064 Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension Naoya Inoue♣,♠, Harsh Trivedi♣, Steven Sinha♣, Niranjan Balasubramanian♣, Kentaro Inui♦,♠ ♣ Stony Brook University, ♠ RIKEN ♦ Tohoku University {ninoue,hjtrivedi,stsinha,niranjan}@cs.stonybrook.edu inui@tohoku.ac.jp Abstract How can we generate concise explanations for multi-hop Reading Comprehension (RC)? The current strategies of identifying supporting sentences can be seen as an extractive question- focused summarization of the input text. How- ever, these extractive explanations are not nec- essarily concise i.e. not minimally sufficient for answering a question. Instead, we advo- cate for an abstractive approach, where we propose to generate a question-focused, ab- stractive summary of input paragraphs and then feed it to an RC system. Given a lim- ited amount of human-annotated abstractive explanations, we train the abstractive explainer in a semi-supervised manner, where we start from the supervised model and then train it further through trial and error maximizing a conciseness-promoted reward function. Our experiments demonstrate that the proposed ab- stractive explainer can generate more compact explanations than an extractive explainer with limited supervision (only 2k instances) while maintaining sufficiency. 1 Introduction Recent approaches to multi-hop Reading Compre- hension (RC) have greatly improved its explain- ability, models ability to explain their own an- swers (Thayaparan et al., 2020). Some adopt a pipelined architecture, where they generate an ex- planation first and then use it to answer the question. This “faithful-by-construction” approach is aimed at ensuring that generated explanations are closer to the systems’ internal reasoning (i.e. faithfulness). The explanation generation step is typically formu- lated as a sentence selection task over the input text — selecting a set of sentences which provide support for the answer output by the model (Yang et al., 2018; Groeneveld et al., 2020, etc.). 1Our implementation is publicly available at https:// github.com/StonyBrookNLP/suqa. Charlie Rowe plays Billy Costa in a film based on what novel? [P1] [1] The Golden Compass is a 2007 British-American fantasy adventure film based on "Northern Lights", the first novel in Philip Pullman's trilogy "His Dark Materials". [2] Written and directed by Chris Weitz, it stars Nicole Kidman, Dakota Blue Richards, Daniel Craig, Sam Elliott, Eva Green, and ... [P2] [1] Charles John Rowe is an English actor. [2] His film roles include Young Tommy in "Never Let Me Go", (...) Billy Costa in "The Golden Compass" , Peter in the SyFy/Sky Movies Peter Pan prequel "Neverland", and recently played Leo Roth on the Fox medical comedy-drama series "Red Band ... ▶ 183 words Charlie Rowe plays Billy Costa in The Golden Compass. The Golden Compass is a film based on the novel Northern Lights. ▶ 21 words Question Northern Lights [P1-1] The Golden Compass is a 2007 British-American fantasy adventure film based on “North... [P2-2] His film roles include Young Tommy in "Never Let Me Go", (...) Billy Costa in “The… ▶ 77 words Paragraphs (a) Extractive Explainer (previous work) (b) Abstractive Explainer (this …

Content truncated to 500 words