Liputan6: A Large-scale Indonesian Dataset for Text Summarization

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 598–608 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 598 Liputan6: A Large-scale Indonesian Dataset for Text Summarization Fajri Koto Jey Han Lau Timothy Baldwin School of Computing and Information Systems The University of Melbourne ffajri@student.unimelb.edu.au, jeyhan.lau@gmail.com, tbaldwin@unimelb.edu.au Abstract In this paper, we introduce a large-scale In- donesian summarization dataset. We har- vest articles from Liputan6.com, an online news portal, and obtain 215,827 document– summary pairs. We leverage pre-trained lan- guage models to develop benchmark extrac- tive and abstractive summarization methods over the dataset with multilingual and mono- lingual BERT-based models. We include a thorough error analysis by examining machine- generated summaries that have low ROUGE scores, and expose both issues with ROUGE it- self, as well as with extractive and abstractive summarization models. 1 Introduction Despite having the fourth largest speaker popula- tion in the world, with 200 million native speakers,1 Indonesian is under-represented in NLP. One rea- son is the scarcity of large datasets for different tasks, such as parsing, text classification, and sum- marization. In this paper, we attempt to bridge this gap by introducing a large-scale Indonesian corpus for text summarization. Neural models have driven remarkable progress in summarization in recent years, particularly for abstractive summarization. One of the first studies was Rush et al. (2015), where the authors proposed an encoder–decoder model with attention to gen- erate headlines for English Gigaword documents (Graff et al., 2003). Subsequent studies introduced pointer networks (Nallapati et al., 2016b; See et al., 2017), summarization with content selection (Hsu et al., 2018; Gehrmann et al., 2018), graph-based attentional models (Tan et al., 2017), and deep re- inforcement learning (Paulus et al., 2018). More recently, we have seen the widespread adoption 1https://www.visualcapitalist.com/ 100-most-spoken-languages/. of pre-trained neural language models for summa- rization, e.g. BERT (Liu and Lapata, 2019), BART (Lewis et al., 2020), and PEGASUS (Zhang et al., 2020a). Progress in summarization research has been driven by the availability of large-scale En- glish datasets, including 320K CNN/Daily Mail document–summary pairs (Hermann et al., 2015) and 100k NYT articles (Sandhaus, 2008) which have been widely used in abstractive summariza- tion research (See et al., 2017; Gehrmann et al., 2018; Paulus et al., 2018; Lewis et al., 2020; Zhang et al., 2020a). News articles are a natural candi- date for summarization datasets, as they tend to be well-structured and are available in large volumes. More recently, English summarization datasets in other flavours/domains have been developed, e.g. XSum has 226K documents with highly abstractive summaries (Narayan et al., 2018), BIGPATENT is a summarization dataset for the legal domain (Sharma et al., 2019), Reddit TIFU is sourced from social media (Kim et al., 2019), and Cohan et al. (2018) proposed using scientific publications from arXiv and PubMed for abstract summarization. This paper introduces the first large-scale sum- marization dataset for Indonesian, sourced from the Liputan6.com online news portal …

Content truncated to 500 words