Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 351–362 August 1–6, 2021. ©2021 Association for Computational Linguistics 351 Entity-Aware Abstractive Multi-Document Summarization Hao Zhou1 , Weidong Ren1 , Gongshen Liu1∗, Bo Su1 , Wei Lu2 1 School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University 2 StatNLP Research Group, Singapore University of Technology and Design {zhou1998,renweidong1997,lgshen,subo}@sjtu.edu.cn luwei@sutd.edu.sg Abstract Entities and their mentions convey significant semantic information in documents. In multi- document summarization, the same entity may appear across different documents. Capturing such cross-document entity information can be beneficial – intuitively, it allows the system to aggregate diverse useful information around the same entity for better summarization. In this paper, we present EMSum, an entity- aware model for abstractive multi-document summarization. Our model augments the classical Transformer-based encoder-decoder framework with a heterogeneous graph con- sisting of text units and entities as nodes, which allows rich cross-document information to be captured. In the decoding process, we design a novel two-level attention mechanism, allowing the model to deal with saliency and redundancy issues explicitly. Our model can also be used together with pre-trained lan- guage models, arriving at improved perfor- mance. We conduct comprehensive experi- ments on the standard datasets and the results show the effectiveness of our approach. 1 Introduction Multi-document summarization aims at generating a short and informative summary across a set of topic-related documents. It is a task that can be more challenging than single-document summa- rization due to the presence of diverse and poten- tially conflicting information (Ma et al., 2020). While significant progress has been made in single-document summarization, the mainstream sequence-to-sequence models, which can perform well on single-document summarization, often struggle with extracting salient information and handling redundancy in the presence of multiple, long documents. Thus, simply adopting models ∗Corresponding author. Figure 1: A sample article from MultiNews. We show the results of co-reference resolution. Mentions of the same entity are highlighted with the same color. that were shown effective for single-document sum- marization to the multi-document setup may not lead to ideal results (Lebanoff et al., 2018; Zhang et al., 2018; Baumel et al., 2018). Several previous research efforts have shown that modeling cross-document relations is essential in multi-document summarization (Liu and Lapata, 2019a; Li et al., 2020). Such relations were shown useful in identifying the salient and redundant infor- mation from long documents, and can thus guide the summary generation process. However, while effective empirically, such approaches do not fo- cus on explicitly modeling the underlying semantic information across documents. Entities and their mentions convey rich semantic information, and can be significant in summariza- tion, especially when a specific entity is the topic under discussion for a set of documents. As shown in Figure 1, entity mentions frequently appear in the input article, and are playing unique roles that contribute towards the coherence and conciseness of the text. We believe that entities can be regarded 352 as the indicator of saliency and can be used to re- duce redundancy. This motivates …

Content truncated to 500 words