KLearn: Background Knowledge Inference from Summarization Data

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2073–2085 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2073 KLearn: Background Knowledge Inference from Summarization Data Maxime Peyrard EPFL maxime.peyrard@epfl.ch Robert West EPFL robert.west@epfl.ch Abstract The goal of text summarization is to com- press documents to the relevant information while excluding background information al- ready known to the receiver. So far, summa- rization researchers have given considerably more attention to relevance than to background knowledge. In contrast, this work puts back- ground knowledge in the foreground. Build- ing on the realization that the choices made by human summarizers and annotators contain implicit information about their background knowledge, we develop and compare tech- niques for inferring background knowledge from summarization data. Based on this frame- work, we define summary scoring functions that explicitly model background knowledge, and show that these scoring functions fit hu- man judgments significantly better than base- lines. We illustrate some of the many po- tential applications of our framework. First, we provide insights into human information importance priors. Second, we demonstrate that averaging the background knowledge of multiple, potentially biased annotators or cor- pora greatly improves summary-scoring per- formance. Finally, we discuss potential appli- cations of our framework beyond summariza- tion. 1 Introduction Summarization is the process of identifying the most important information pieces in a document. For humans, this process is heavily guided by back- ground knowledge, which encompasses preconcep- tions about the task and priors about what kind of information is important (Mani, 1999). Despite its fundamental role, background knowl- edge has received little attention from the summa- rization community. Existing approaches largely focus on the relevance aspect, which enforces sim- ilarity between the generated summaries and the source documents (Peyrard, 2019). Unobserved Observed ∘ Background Knowledge (K) Document (D) Summary (S) >>> <<< <<< >>> >>> <<< <<< >>> : Should be similar: small KL(S||D) (Relevance) : Should be different: large KL(S||K) (Informativeness) Figure 1: A summary (S) results from the combina- tion of the background knowledge (K) and the source document (D). Following Peyrard (2019), S is simi- lar to D (Relevance measured by a small KL(S||D)) but also brings new information compared to back- ground knowledge (informativeness measured by a large KL(S||K)). We can infer the unobserved K from the choices unexplained by the Relevance criteria. In previous work, background knowledge has usually been modeled by simple aggregation of large background corpora. For instance, using TF·IDF (Sparck Jones, 1972), one may operational- ize background knowledge as the set of words with a large document frequency in background corpora. However, the assumption that frequently dis- cussed topics reflect what is, on average, known does not necessarily hold. For example, common- sense information is often not even discussed (Liu and Singh, 2004). Also, information present in background texts has already gone through the im- portance filter of humans, e.g., writers and publish- ers. In general, a particular difficulty preventing the development of proper background knowledge models is its latent nature. We …

Content truncated to 500 words