Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6718–6724 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6718 Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation Paul Tardy1, 2 David Janiszek 1, 3 Yannick Estève 4 Vincent Nguyen 2 1 LIUM – Le Mans Université 2 Ubiqus Labs 3 Université Paris Descartes 4 LIA – Avignon Université pltrdy@gmail.com, david.janiszek@parisdescartes.fr, yannick.esteve@univ-avignon.fr, vnguyen@ubiqus.com Abstract Summarizing texts is not a straightforward task. Before even considering text summarization, one should determine what kind of summary is expected. How much should the information be compressed? Is it relevant to reformulate or should the summary stick to the original phrasing? State-of-the-art on automatic text summarization mostly revolves around news articles. We suggest that considering a wider variety of tasks would lead to an improvement in the field, in terms of generalization and robustness. We explore meeting summarization: generating reports from automatic transcriptions. Our work consists in segmenting and aligning transcriptions with respect to reports, to get a suitable dataset for neural summarization. Using a bootstrapping approach, we provide pre-alignments that are corrected by human annotators, making a validation set against which we evaluate automatic models. This consistently reduces annotators’ efforts by providing iteratively better pre-alignment and maximizes the corpus size by using annotations from our automatic alignment models. Evaluation is conducted on public_meetings, a novel corpus of aligned public meetings. We report automatic alignment and summarization performances on this corpus and show that automatic alignment is relevant for data annotation since it leads to large improvement of almost +4 on all ROUGE scores on the summarization task. Keywords: Alignment, Summarization, Corpus Annotation 1. Introduction Automatic Text Summarization is the task of producing a short text that captures the most salient points of a longer one. However, a large variety of tasks could fit this defini- tion. Many factors are critical in the summarization process, such as whether to rephrase the source (abstractiveness) or use part of the source as-is (extractiveness); the length ra- tio of target and source (compression factor); the source and target lengths and their variances; and the information distribution – i.e. how important information is distributed along the text. Most of summarization benchmarks (See et al., 2017; Paulus et al., 2017; Gehrmann et al., 2018) rely on news articles from CNN and DailyMail (Hermann et al., 2015; Nallapati et al., 2016) which exhibit particular character- istics such as: (i) being quite extractive i.e. picking por- tions of text from the source, the opposite of abstractive (Liu et al., 2018); (ii) a high compression factor with the summary being up to 10 times shorter than the source (Liu et al., 2018); (iii) a low variance in both source and target length, and (iv) concentrating information mostly at the be- ginning of the article: for example, papers working on the CNN-DailyMail corpus (Hermann et al., 2015; Nallapati et al., 2016) often truncate the article to the first 400 words of the article (See …

Content truncated to 500 words