A Simple Theoretical Model of Importance for Summarization

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1059–1073 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 1059 A Simple Theoretical Model of Importance for Summarization Maxime Peyrard∗ EPFL maxime.peyrard@epfl.ch Abstract Research on summarization has mainly been driven by empirical approaches, crafting sys- tems to perform well on standard datasets with the notion of information Importance remain- ing latent. We argue that establishing the- oretical models of Importance will advance our understanding of the task and help to fur- ther improve summarization systems. To this end, we propose simple but rigorous defi- nitions of several concepts that were previ- ously used only intuitively in summarization: Redundancy, Relevance, and Informativeness. Importance arises as a single quantity naturally unifying these concepts. Additionally, we pro- vide intuitions to interpret the proposed quan- tities and experiments to demonstrate the po- tential of the framework to inform and guide subsequent works. 1 Introduction Summarization is the process of identifying the most important information from a source to pro- duce a comprehensive output for a particular user and task (Mani, 1999). While producing readable outputs is a problem shared with the field of Nat- ural Language Generation, the core challenge of summarization is the identification and selection of important information. The task definition is rather intuitive but involves vague and undefined terms such as Importance and Information. Since the seminal work of Luhn (1958), au- tomatic text summarization research has focused on empirical developments, crafting summariza- tion systems to perform well on standard datasets leaving the formal definitions of Importance la- tent (Das and Martins, 2010; Nenkova and McKe- own, 2012). This view entails collecting datasets, defining evaluation metrics and iteratively select- ing the best-performing systems either via super- ∗Research partly done at UKP Lab from TU Darmstadt. vised learning or via repeated comparison of un- supervised systems (Yao et al., 2017). Such solely empirical approaches may lack guidance as they are often not motivated by more general theoretical frameworks. While these ap- proaches have facilitated the development of prac- tical solutions, they only identify signals correlat- ing with the vague human intuition of Importance. For instance, structural features like centrality and repetitions are still among the most used proxies for Importance (Yao et al., 2017; Kedzie et al., 2018). However, such features just correlate with Importance in standard datasets. Unsurprisingly, simple adversarial attacks reveal their weaknesses (Zopf et al., 2016). We postulate that theoretical models of Impor- tance are beneficial to organize research and guide future empirical works. Hence, in this work, we propose a simple definition of information im- portance within an abstract theoretical framework. This requires the notion of information, which has received a lot of attention since the work from Shannon (1948) in the context of communication theory. Information theory provides the means to rigorously discuss the abstract concept of informa- tion, which seems particularly well suited as an entry point for a theory of summarization. How- ever, information theory concentrates on …

Content truncated to 500 words