Abstract Text Summarization: A Low Resource Challenge

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5994–5998, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5994 Abstract Text Summarization: A Low Resource Challenge Shantipriya Parida Petr Motlicek Idiap Research Institute Rue Marconi 19, 1920 Martigny, Switzerland {shantipriya.parida,petr.motlicek}@idiap.ch Abstract Text summarization is considered as a chal- lenging task in the NLP community. The avail- ability of datasets for the task of multilingual text summarization is rare, and such datasets are difficult to construct. In this work, we build an abstract text summarizer for the Ger- man language text using the state-of-the-art “Transformer” model. We propose an itera- tive data augmentation approach which uses synthetic data along with the real summariza- tion data for the German language. To gener- ate synthetic data, the Common Crawl (Ger- man) dataset is exploited, which covers dif- ferent domains. The synthetic data is effec- tive for the low resource condition and is par- ticularly helpful for our multilingual scenario where availability of summarizing data is still a challenging issue. The data are also useful in deep learning scenarios where the neural mod- els require a large amount of training data for utilization of its capacity. The obtained sum- marization performance is measured in terms of ROUGE and BLEU score. We achieve an absolute improvement of +1.5 and +16.0 in ROUGE1 F1 (R1 F1) on the development and test sets, respectively, compared to the system which does not rely on data augmentation. 1 Introduction Automatic text summarization is considered as a challenging task because while summarizing a piece of text, we read it entirely to develop our understanding to prepare highlighting its main points. Due to the lack of human knowledge and language processing abilities in computers, auto- matic text summarization is a major non-trivial task (Allahyari et al., 2017). Two major approaches for automatic summa- rization are: extractive and abstractive. The ex- tractive summarization approach produces sum- maries by choosing a subset of sentences in the original text. The abstract text summarization ap- proach aims to shorten the long text into a human- readable form that contains the most important fact from the original text (Allahyari et al., 2017; Kry´sci´nski et al., 2018). The deep learning-based neural attention model when applying to abstract text summarization per- forms well compared to standard learning-based approaches (Rush et al., 2015). Abstract text sum- marization using the attentional encoder-decoder recurrent neural network approach shows a state- of-the-art performance and sets a baseline model (Nallapati et al., 2016). Further improvements are introduced to the baseline model by using the pointer generator network and coverage mecha- nism using reinforcement learning based training procedure (See et al., 2017; Paulus et al., 2017). There is an inherent limitation to natural lan- guage processing tasks such as text summariza- tion for resource-poor and morphological complex languages owing to a shortage of quality linguis- tic data available (Kurniawan and Louvan, 2018). The use of synthetic data …

Content truncated to 500 words