What’s New? Summarizing Contributions in Scientific Literature

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1019–1031 May 2-6, 2023 ©2023 Association for Computational Linguistics What’s New? Summarizing Contributions in Scientific Literature Hiroaki Hayashi1 Wojciech Kry´sci´nski1 Bryan McCann2 Nazneen Rajani3 Caiming Xiong1 1Salesforce Research 2You.com 3Hugging Face hiroakihayashi@salesforce.com Abstract With thousands of academic articles shared on a daily basis, it has become increasingly diffi- cult to keep up with the latest scientific find- ings. To overcome this problem, we introduce a new task of disentangled paper summarization, which seeks to generate separate summaries for the paper contributions and the context of the work, making it easier to identify the key findings shared in articles. For this purpose, we extend the S2ORC corpus of academic articles, which spans a diverse set of domains ranging from economics to psychology, by adding dis- entangled “contribution” and “context” refer- ence labels. Together with the dataset, we in- troduce and analyze three baseline approaches: 1) a unified model controlled by input code prefixes, 2) a model with separate generation heads specialized in generating the disentan- gled outputs, and 3) a training strategy that guides the model using additional supervision coming from inbound and outbound citations. We also propose a comprehensive automatic evaluation protocol which reports the relevance, novelty, and disentanglement of generated out- puts. Through a human study involving expert annotators, we show that in 79%, of cases our new task is considered more helpful than tradi- tional scientific paper summarization. 1 Introduction With the growing popularity of open-access aca- demic article repositories, such as arXiv or bioRxiv, disseminating new research findings has become nearly effortless. Through such services, tens of thousands of scientific papers are shared by the re- search community every month1. At the same time, the unreviewed nature of mentioned repositories and the sheer volume of new publications has made it nearly impossible to identify relevant work and keep up with the latest findings. 1https://arxiv.org/stats/monthly_submissions Scientific paper summarization, a subtask within automatic text summarization, aims to assist re- searchers in their work by automatically condens- ing articles into a short, human-readable form that contains only the most essential information. In re- cent years, abstractive summarization, an approach where models are trained to generate fluent sum- maries by paraphrasing the source article, has seen impressive progress. State-of-the-art methods lever- age large, pre-trained models (Lewis et al., 2020; Liu et al., 2022), define task-specific pre-training strategies (Zhang et al., 2019), and scale to long input sequences (Zhao et al., 2020; Zaheer et al., 2020). Available large-scale benchmark datasets, such as arXiv and PubMed (Cohan et al., 2018), were automatically collected from online archives and repurpose paper abstracts as reference sum- maries. However, the current form of scientific paper summarization where models are trained to generate paper abstracts has two caveats: 1) of- ten, abstracts contain information which is not of primary importance, 2) the vast majority of scien- tific articles come with human-written abstracts, making the generated summaries superfluous. To address these shortcomings, we …

Content truncated to 500 words