A Multi-level Annotated Corpus of Scientific Papers for Scientific Document Summarization and Cross-document Relation Discovery

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6672–6679 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6672 A Multi-level Annotated Corpus of Scientific Papers for Scientific Document Summarization and Cross-document Relation Discovery Ahmed AbuRa’ed1, Horacio Saggion1, Luis Chiruzzo2 1Large Scale Text Understanding Systems Lab TALN Research Group Universitat Pompeu Fabra, Barcelona, Spain 2Universidad de la Rep´ublica, Montevideo, Uruguay ahmed.aburaed@upf.edu, horacio.saggion@upf.edu, luischir@fing.edu.uy Abstract Related work sections or literature reviews are an essential part of every scientific article being crucial for paper reviewing and assessment. The automatic generation of related work sections can be considered an instance of the multi-document summarization problem. In order to allow the study of this specific problem, we have developed a manually annotated, machine readable data-set of related work sections, cited papers (e.g. references) and sentences, together with an additional layer of papers citing the references. We additionally present experiments on the identification of cited sentences, using as input citation contexts. The corpus alongside the gold standard are made available for use by the scientific community. Keywords: Human Annotated Corpora, Text Summarization, Cross-document Relations, Related Work Sections, Literature Re- views 1. Introduction Most scientific papers include a related work section providing, in a well organized and condensed form, the key information from a carefully selected list of pub- lications which contextualize and ground the research being presented by an author (Rowley and Slack, 2004). Related work sections are critical for quality assessment since journals pay particular attention to them where evaluation of manuscripts is of concern (Maggio et al., 2016). Past research has shown that related work sections can be produced following cut-and-paste summarization strategies (Jaidka et al., 2013) which are typical of doc- ument abstracting (insertion, deletion, substitution, etc.) (Endres-Niggemeyer et al., 1995; Saggion, 2011). Recent studies have proposed to take advantage of the scientific paper’s citation network to approach scientific literature summarization. For that reason we introduce here our corpus which we hope will facilitate the usage of citation networks to boost scientific literature summariza- tion research. The generation of related work sections has been studied from different viewpoints (Hoang and Kan, 2010; Vu, 2010; Hu and Wan, 2014), however no manual annotated data-set, analog to the one we will present here, has been produced until now. Our corpus expands considerably the data-set of related work sections used in (Hoang and Kan, 2010) by providing: (i) related work sections, (ii) a manually annotated layer of cited papers and sentences, (iii) citing papers referring to the cited papers in the related work section, and (iv) a layer of rich linguistic, rhetorical, and semantic annotations computed automatically. While the manually identified cited sentences are useful to support the study of sequence to sequence models in scientific summarization, the new layer of citing papers facilitates the test of citation-based summarization approaches (Qazvinian and Radev, 2008; Jaidka et al., 2014b) which rely on citation networks to assess sentence relevance. In this corpus we refer to three types of …

Content truncated to 500 words