Entity Commonsense Representation for Neural Abstractive Summarization

Proceedings of NAACL-HLT 2018, pages 697–707 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Entity Commonsense Representation for Neural Abstractive Summarization Reinald Kim Amplayo∗and Seonjae Lim∗and Seung-won Hwang Yonsei University, Seoul, South Korea {rktamplayo, sun.lim, seungwonh}@yonsei.ac.kr Abstract A major proportion of a text summary includes important entities found in the original text. These entities build up the topic of the sum- mary. Moreover, they hold commonsense in- formation once they are linked to a knowledge base. Based on these observations, this pa- per investigates the usage of linked entities to guide the decoder of a neural text summarizer to generate concise and better summaries. To this end, we leverage on an off-the-shelf entity linking system (ELS) to extract linked entities and propose Entity2Topic (E2T), a module easily attachable to a sequence-to-sequence model that transforms a list of entities into a vector representation of the topic of the sum- mary. Current available ELS’s are still not suf- ficiently effective, possibly introducing unre- solved ambiguities and irrelevant entities. We resolve the imperfections of the ELS by (a) en- coding entities with selective disambiguation, and (b) pooling entity vectors using firm atten- tion. By applying E2T to a simple sequence- to-sequence model with attention mechanism as base model, we see significant improve- ments of the performance in the Gigaword (sentence to title) and CNN (long document to multi-sentence highlights) summarization datasets by at least 2 ROUGE points. 1 Introduction Text summarization is a task to generate a shorter and concise version of a text while preserving the meaning of the original text. The task can be di- vided into two subtask based on the approach: ex- tractive and abstractive summarization. Extrac- tive summarization is a task to create summaries by pulling out snippets of text form the origi- nal text and combining them to form a summary. Abstractive summarization asks to generate sum- maries from scratch without the restriction to use ∗ Amplayo and Lim are co-first authors with equal con- tribution. Names are arranged alphabetically. The Los Angeles Dodgers acquired South Korean right-hander Jae Seo from the New York Mets on Wednesday in a four-player swap. Input Text Korea’s Seo headed to Dodgers from Mets Summary Topic: Entity Distribution  Los Angeles Dodgers -> /wiki/Los_Angeles_Dodgers  South Korean -> /wiki/South_Korean  Jae Seo -> /wiki/Seo_Jae-woong  New York Mets -> /wiki/New_York_Mets  Wednesday -> /wiki/Wednesday_Night_Baseball  swap -> /wiki/Trade_(sports) 0.4 0.0 0.3 0.2 0.1 0.0 O1 O2 O3 Figure 1: Observations on linked entities in summaries. O1: Summaries are mainly composed of entities. O2: Entities can be used to represent the topic of the sum- mary. O3: Entity commonsense learned from a large corpus can be used. the available words from the original text. Due to the limitations of extractive summarization on incoherent texts and unnatural methodology (Yao et al., 2017), the research trend has shifted towards abstractive summarization. Sequence-to-sequence models (Sutskever et al., 2014) with attention mechanism (Bahdanau et al., 2014) have found great success in …

Content truncated to 500 words