Discourse-Aware Hierarchical Attention Network for Extractive Single-Document Summarization

Proceedings of Recent Advances in Natural Language Processing, pages 497–506, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_059 497 Discourse-Aware Hierarchical Attention Network for Extractive Single-Document Summarization Tatsuya Ishigaki⋆ Hidetaka Kamigaito⋆ Hiroya Takamura⋆⋄ Manabu Okumura⋆ ⋆Tokyo Institute of Technology ⋄National Institute of Advanced Industrial Science and Technology {ishigaki, kamigaito}@lr.pi.titech.ac.jp, {takamura, oku}@pi.titech.ac.jp Abstract Discourse relations between sentences are often represented as a tree, and the tree structure provides important information for summarizers to create a short and co- herent summary. However, current neu- ral network-based summarizers treat the source document as just a sequence of sentences and ignore the tree-like dis- course structure inherent in the docu- ment. To incorporate the information of a discourse tree structure into the neural network-based summarizers, we propose a discourse-aware neural extractive summa- rizer which can explicitly take into account the discourse dependency tree structure of the source document. Our discourse- aware summarizer can jointly learn the discourse structure and the salience score of a sentence by using novel hierarchical attention modules, which can be trained on automatically parsed discourse depen- dency trees. Experimental results showed that our model achieved competitive or better performances against state-of-the- art models in terms of ROUGE scores on the DailyMail dataset. We further con- ducted manual evaluations. The results showed that our approach also gained the coherence of the output summaries. 1 Introduction Document summarization is the task of automati- cally shortening a source document while retain- ing its salient information. In this paper, we present a recurrent neural network (RNN)-based extractive summarizer taking into account the dis- course structure inherent in the source document. S1: A London-based football team West Ham wants to take Chelsea winger Christian Atsu on loan. S2: The 22-year-old was signed from Porto last year but loaned out to Vitesse Arnhem. S3: Aston Villa is also interested in him. S4: Meanwhile , West Ham also has bid 12million for Ecuador ’s Enner Valencia. ROOT Figure 1: Example of discourse dependency struc- ture. The discourse structure consists of discourse re- lations between units in the input, and discourse information has been shown useful for summa- rization tasks. An example of a Rhetorical Struc- ture Theory (RST) (Mann and Thompson, 1988)- based discourse structure, expressed as a depen- dency tree, is illustrated in Figure 1. In the figure, each node corresponds to a sentence. Regarding the relations between the sentences, sentence S2 elaborates the fact mentioned in sentence S1. In addition, S2 is further elaborated by S3. S4 is a contrast to the mention S1. Such relations are es- sential cues for generating a concise and coherent summary. For example, elaborated sentences tend to be more important than elaborating sentences, and the elaborated sentences should be included in the summary while the elaborating sentences are not. Several Integer Linear Programming (ILP)- based summarizers (Hirao et al., 2013; Kikuchi et al., 2014) use the discourse information given by a discourse parser (Hernault et al., 2010). Thus, the performance of the summarizers is strongly af- fected by the performance of …

Content truncated to 500 words