TalkSumm: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2125–2131 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 2125 TALKSUMM: A Dataset and Scalable Annotation Method for Scientific Paper Summarization Based on Conference Talks Guy Lev∗, Michal Shmueli-Scheuer∗, Jonathan Herzig, Achiya Jerbi, David Konopnicki IBM Research, Haifa, Israel {guylev,shmueli,hjon,davidko}@il.ibm.com, achiya.jerbi@ibm.com Abstract Currently, no large-scale training data is avail- able for the task of scientific paper sum- marization. In this paper, we propose a novel method that automatically generates summaries for scientific papers, by utilizing videos of talks at scientific conferences. We hypothesize that such talks constitute a coher- ent and concise description of the papers’ con- tent, and can form the basis for good sum- maries. We collected 1716 papers and their corresponding videos, and created a dataset of paper summaries. A model trained on this dataset achieves similar performance as mod- els trained on a dataset of summaries created manually. In addition, we validated the quality of our summaries by human experts. 1 Introduction The rate of publications of scientific papers is increasing and it is almost impossible for re- searchers to keep up with relevant research. Au- tomatic text summarization could help mitigate this problem. In general, there are two com- mon approaches to summarizing scientific papers: citations-based, based on a set of citation sen- tences (Nakov et al., 2004; Abu-Jbara and Radev, 2011; Yasunaga et al., 2019), and content-based, based on the paper itself (Collins et al., 2017; Nikola Nikolov and Hahnloser, 2018). Auto- matic summarization is studied exhaustively for the news domain (Cheng and Lapata, 2016; See et al., 2017), while summarization of scientific pa- pers is less studied, mainly due to the lack of large- scale training data. The papers’ length and com- plexity require substantial summarization effort from experts. Several methods were suggested to reduce these efforts (Yasunaga et al., 2019; Collins et al., 2017), still they are not scalable as they re- quire human annotations. ∗ The authors contributed equally. Title: Split and Rephrase: Better Evaluation and Stronger Baselines (Aharoni and Goldberg, 2018) Paper: Processing long, complex sentences is challenging. This is true either for humans in various circumstances or in NLP tasks like parsing and machine translation . An automatic system capable of breaking a complex sentence into several simple sentences that convey the same meaning is very appealing . A recent work by Narayan et al. (2017) introduced a dataset, evaluation method and baseline systems for the task, naming it Split-and Rephrase . The dataset includes 1,066,115 instances mapping a single complex sentence to a sequence of sentences that express the same meaning, together with RDF triples that describe their semantics. They considered two . .. Indeed, feeding the model with examples containing entities alone without any facts about them causes it to output perfectly phrased but unsupported facts (Table 3). Digging further, we find that 99% of the simple sentences (more than 89% of the unique ones) in the validation and …

Content truncated to 500 words