S2ORC: The Semantic Scholar Open Research Corpus

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4969–4983 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4969 S2ORC: The Semantic Scholar Open Research Corpus Kyle Lo†∗ Lucy Lu Wang†∗ Mark Neumann† Rodney Kinney† Daniel S. Weld†‡ †Allen Institute for Artificial Intelligence ‡Paul G. Allen School of Computer Science & Engineering, University of Washington {kylel, lucyw}@allenai.org Abstract We introduce S2ORC,1 a large corpus of 81.1M English-language academic papers spanning many academic disciplines. The cor- pus consists of rich metadata, paper abstracts, resolved bibliographic references, as well as structured full text for 8.1M open access pa- pers. Full text is annotated with automatically- detected inline mentions of citations, figures, and tables, each linked to their correspond- ing paper objects. In S2ORC, we aggregate papers from hundreds of academic publishers and digital archives into a unified source, and create the largest publicly-available collection of machine-readable academic text to date. We hope this resource will facilitate research and development of tools and tasks for text mining over academic text. 1 Introduction Academic papers are an increasingly important textual domain for natural language processing (NLP) research. Aside from capturing valuable knowledge from humankind’s collective research efforts, academic papers exhibit many interest- ing characteristics – thousands of words organized into sections, objects such as tables, figures and equations, frequent inline references to these ob- jects, footnotes, other papers, and more. Different types of resources have been used to support research over academic papers. Cita- tion graphs like AMiner’s Open Academic Graph (Tang et al., 2008), the Microsoft Academic Graph (MAG) (Shen et al., 2018), and the Semantic Scholar literature graph (Ammar et al., 2018), have had widespread application in bibliomet- rics, science-of-science, information retrieval, and network analysis. Digital archives like arXiv,2 ∗denotes equal contribution 1Instructions for access to the data and model are avail- able at https://github.com/allenai/s2orc/. 2https://arxiv.org Figure 1: Inline citations and references to figures and tables are annotated in S2ORC’s structured full text. Citations are linked to bibliography entries, which are linked to other papers in S2ORC. Figure and table ref- erences are linked to their captions. PubMed Central,3 CiteSeerX (Giles et al., 1998),4 and the ACL Anthology (Bird et al., 2008),5 are popular resources for deriving large text cor- pora for summarization and language modeling or, with further annotation, development of datasets for tasks like entity extraction, text classifica- tion, parsing, and discourse analysis. We focus on bibliometrically-enhanced derivations of these corpora, such as the ACL Anthology Network (AAN) (Radev et al., 2009)6 derived from the ACL Anthology, RefSeer (Huang et al., 2015) derived from CiteSeerX, and Saier and F¨arber (2019) de- rived from arXiv, which combine useful aspects of citation graphs and raw text corpora. These re- sources provide citation mentions linked to paper identifiers in their corresponding digital archives, such as the ACL Anthology and CiteSeerX, or to nodes in citation graphs such as MAG, enabling new forms of cross-paper discourse analysis (e.g., studying how or why papers are related). 3https://www.ncbi.nlm.nih.gov/pmc 4https://citeseerx.ist.psu.edu 5https://www.aclweb.org/anthology …

Content truncated to 500 words