ProDial – An Annotated Proactive Dialogue Act Corpus for Conversational Assistants using Crowdsourcing

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 313–318 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 313 Learning to Prioritize: Precision-Driven Sentence Filtering for Long Text Summarization Alex Mei, Anisha Kabir, Rukmini Bapat, John Judge, Tony Sun, William Yang Wang University of California, Santa Barbara, Santa Barbara, CA {alexmei, anishakabir, rbapat, jjudge, tonysun}@ucsb.edu, william@cs.ucsb.edu Abstract Neural text summarization has shown great potential in recent years. However, current state-of-the-art summarization models are limited by their maximum input length, posing a challenge to summarizing longer texts comprehensively. As part of a layered summarization architecture, we introduce PURETEXT, a simple yet effective pre-processing layer that removes low- quality sentences in articles to improve existing summarization models. When evaluated on popular datasets like WikiHow and Reddit TIFU, we show up to 3.84 and 8.57 point ROUGE-1 absolute improvement on the full test set and the long article subset, respectively, for state-of-the-art summarization models such as BERTSUM and BART. Our approach provides downstream models with higher-quality sentences for summarization, improving overall model performance, especially on long text articles. Keywords: text summarization, machine learning, natural language processing 1. Introduction Neural summarization models have evolved quickly over time, successfully tackling increasingly complex problems relating to natural language (Zhong et al., 2020; Zhou et al., 2018; Zhang et al., 2019; Xu et al., 2020). One key problem that has plagued state-of- the-art summarization models is their maximum input length (Liu, 2019; Lewis et al., 2020). Although recent work has progressed towards addressing this issue for Transformer-based models (Beltagy et al., 2020; Zhou et al., 2021; Choromanski et al., 2021), not as much attention has been paid specifically to long text sum- marization. Summarization models such as BERTSUM (Liu, 2019) and BART (Lewis et al., 2020) either truncate or cannot handle articles longer than the maximum input length. Truncation may leave out critical parts of the text, lead- ing to an incomplete summary. For datasets where LEAD-3 forms a decent baseline (Nallapati et al., 2016; Narayan et al., 2018), truncat- ing an article’s ending may not greatly affect summa- rization. While this may be true for news summariza- tion datasets in which story highlights tend to appear at the start (Hermann et al., 2015; Nallapati et al., 2016; Narayan et al., 2018), other datasets such as WikiHow (Koupaee and Wang, 2018) and Reddit TIFU (Kim et al., 2019) typically do not follow the same journalis- tic structure. WikiHow instructional texts contain key steps evenly dispersed throughout the article, and Red- dit stories tend to follow a narrative arc where the cli- max is toward the end of the passage. One simple solution to truncation is to omit the middle section of an article instead (Sun et al., 2019). How- ever, this method, along with similar approaches, is a heuristic band-aid solution that can potentially be im- proved upon with a more versatile model. While existing works show promising results for long text summarization (Beltagy et al., 2020; Xu …

Content truncated to 500 words