CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10922 - 10935 December 7-11, 2022 ©2022 Association for Computational Linguistics CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision Yuning Mao, Ming Zhong, Jiawei Han University of Illinois Urbana-Champaign {yuningm2, mingz5, hanj}@illinois.edu Abstract Scientific extreme summarization (TLDR) aims to form ultra-short summaries of scientific pa- pers. Previous efforts on curating scientific TLDR datasets failed to scale up due to the heavy human annotation and domain expertise required. In this paper, we propose a simple yet effective approach to automatically extract- ing TLDR summaries for scientific papers from their citation texts. Based on the proposed ap- proach, we create a new benchmark CiteSum without human annotation, which is around 30 times larger than the previous human-curated dataset SciTLDR. We conduct a comprehen- sive analysis of CiteSum, examining its data characteristics and establishing strong base- lines. We further demonstrate the usefulness of CiteSum by adapting models pre-trained on CiteSum (named CITES) to new tasks and do- mains with limited supervision. For scientific extreme summarization, CITES outperforms most fully-supervised methods on SciTLDR without any fine-tuning and obtains state-of-the- art results with only 128 examples. For news extreme summarization, CITES achieves sig- nificant gains on XSum over its base model (not pre-trained on CiteSum), e.g., +7.2 ROUGE- 1 zero-shot performance and state-of-the-art few-shot performance. For news headline gen- eration, CITES performs the best among unsu- pervised and zero-shot methods on Gigaword.1 1 Introduction Scientific summarization typically regards paper abstract as the ground-truth summary, as it is writ- ten by the authors themselves with relatively high quality and readily available in most scientific docu- ments. However, paper abstract may not always be the ideal summary because it often involves certain details such as task description, background infor- mation, and experiment results (cf. the abstract of this paper). As a result, recent work (Cachola et al., 1Our dataset and code can be found at https:// github.com/morningmoni/CiteSum. Paper Abstract: We study the problem of transferring a sample in one domain to an analog sample in an- other domain. Given two related domains, S and T , we would like to learn a generative function G that maps an input sample from S to the domain T , such that the out- put of a given function f , which accepts inputs in either domains, would remain unchanged. Other than the func- tion f , the training data is unsupervised and consist of a set of samples from each domain. The Domain Trans- fer Network (DTN) we present employs a compound loss function that includes a multiclass GAN loss, an f -constancy component, and a regularizing component that encourages G to map samples from T to themselves. We apply our method to visual domains including digits and face images and demonstrate its ability to generate convincing novel images of previously unseen entities, while preserving their identity. Citation Text: Taigman et al. [8] proposed the Domain Transfer Network (DTN) to map a …

Content truncated to 500 words