Abstractive Summarization of Reddit Posts with Multi-level Memory Networks

Proceedings of NAACL-HLT 2019, pages 2519–2531 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2519 Abstractive Summarization of Reddit Posts with Multi-level Memory Networks Byeongchang Kim Hyunwoo Kim Gunhee Kim Department of Computer Science and Engineering & Center for Superintelligence Seoul National University, Seoul, Korea {byeongchang.kim,hyunwoo.kim}@vision.snu.ac.kr gunhee@snu.ac.kr Abstract We address the problem of abstractive summa- rization in two directions: proposing a novel dataset and a new model. First, we col- lect Reddit TIFU dataset, consisting of 120K posts from the online discussion forum Red- dit. We use such informal crowd-generated posts as text source, in contrast with exist- ing datasets that mostly use formal documents as source such as news articles. Thus, our dataset could less suffer from some biases that key sentences usually locate at the be- ginning of the text and favorable summary candidates are already inside the text in sim- ilar forms. Second, we propose a novel ab- stractive summarization model named multi- level memory networks (MMN), equipped with multi-level memory to store the informa- tion of text from different levels of abstraction. With quantitative evaluation and user stud- ies via Amazon Mechanical Turk, we show the Reddit TIFU dataset is highly abstrac- tive and the MMN outperforms the state-of- the-art summarization models. The code and dataset are available at http://vision. snu.ac.kr/projects/reddit-tifu. 1 Introduction Abstractive summarization methods have been un- der intensive study, yet they often suffer from in- ferior performance compared to extractive meth- ods (Allahyari et al., 2017; Nallapati et al., 2017; See et al., 2017). Admittedly, by task defini- tion, abstractive summarization is more challeng- ing than extractive summarization. However, we argue that such inferior performance is partly due to some biases of existing summarization datasets. The source text of most datasets (Over et al., 2007; Hermann et al., 2015; Cohan et al., 2018; Grusky et al., 2018; Narayan et al., 2018a) originates from formal documents such as news articles, which have some structural patterns of which extractive methods better take advantage. In formal documents, there could be a strong tendency that key sentences locate at the begin- ning of the text and favorable summary candi- dates are already inside the text in similar forms. Hence, summarization methods could generate good summaries by simply memorizing keywords or phrases from particular locations of the text. Moreover, if abstractive methods are trained on these datasets, they may not show much abstrac- tion (See et al., 2017), because they are implicitly forced to learn structural patterns (Kedzie et al., 2018). Grusky et al. (2018) and Narayan et al. (2018a) recently report similar extractive bias in existing datasets. They alleviate this bias by col- lecting articles from diverse news publications or regarding intro sentences as gold summary. Different from previous approaches, we pro- pose to alleviate such bias issue by changing the source of summarization dataset. We exploit user- generated posts from the online discussion forum Reddit, especially TIFU subreddit, which are more casual and conversational than news articles. We observe that the …

Content truncated to 500 words