Generating Query Focused Summaries from Query-Free Resources

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6096–6109 August 1–6, 2021. ©2021 Association for Computational Linguistics 6096 Generating Query Focused Summaries from Query-Free Resources Yumo Xu and Mirella Lapata Institute for Language, Cognition and Computation School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB yumo.xu@ed.ac.uk mlap@inf.ed.ac.uk Abstract The availability of large-scale datasets has driven the development of neural models that create generic summaries from single or mul- tiple documents. In this work we consider query focused summarization (QFS), a task for which training data in the form of queries, doc- uments, and summaries is not readily available. We propose to decompose QFS into (1) query modeling (i.e., finding supportive evidence within a set of documents for a query) and (2) conditional language modeling (i.e., sum- mary generation). We introduce MARGE, a Masked ROUGE Regression framework for evidence estimation and ranking which relies on a unified representation for summaries and queries, so that summaries in generic data can be converted into proxy queries for learning a query model. Experiments across QFS bench- marks and query types show that our model achieves state-of-the-art performance despite learning from weak supervision.1 1 Introduction The neural encoder-decoder framework has be- come increasingly popular in generic summariza- tion (See et al. 2017; Gehrmann et al. 2018; Liu and Lapata 2019a; Fabbri et al. 2019, inter alia) thanks to the availability of large-scale datasets containing hundreds of thousands of document-summary pairs. Training data of this magnitude is not readily avail- able for query focused summarization (QFS; Dang 2005) which aims to create a short summary from a set of documents that answers a specific query. Existing corpora (Nema et al., 2017; Dang, 2005; Hoa, 2006; Baumel et al., 2016) are relatively small for modern data-hungry neural architectures and have been mostly used for evaluation purposes. 1Our code and data is available at https://github. com/yumoxu/marge. A major bottleneck in leveraging generic sum- marization data for QFS is the absence of queries (Nema et al., 2017); the majority of existing datasets consist of document-summary pairs, while QFS summaries are expected to answer specific queries. Recent work (Xu and Lapata, 2020; Su et al., 2020; Laskar et al., 2020) sidesteps this problem by resorting to distant supervision from query-relevant NLP resources including question answering (Rajpurkar et al., 2016; Chakraborty et al., 2020) and paraphrase identification (Dolan and Brockett, 2005). Such approaches incorporate query modeling in the summarization process but are even more data hungry compared to generic summarization ones, since they additionally re- quire access to QA datasets which can be extremely costly to create (Bajaj et al., 2016; Kwiatkowski et al., 2019). Moreover, there is often a mismatch between queries in QA datasets and those in QFS scenarios (Xu and Lapata, 2020); the two types of queries are not identically distributed and it is prac- tically infeasible to find appropriate query-related resources for all domains and topics. In this work …

Content truncated to 500 words