The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2192–2203 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2192 The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources Jennifer D’Souza, Anett Hoppe, Arthur Brack, Mohamad Yaser Jaradeh, S¨oren Auer, Ralph Ewerth TIB Leibniz Information Centre for Science and Technology, Hannover, Germany {jennifer.dsouza,anett.hoppe,arthur.brack,yaser.jaradeh,auer,ralph.ewerth}@tib.eu Abstract We introduce the STEM (Science, Technology, Engineering, and Medicine) Dataset for Scientific Entity Extraction, Classification, and Resolution, version 1.0 (STEM-ECR v1.0). The STEM-ECR v1.0 dataset has been developed to provide a benchmark for the evaluation of scientific entity extraction, classification, and resolution tasks in a domain-independent fashion. It comprises abstracts in 10 STEM disciplines that were found to be the most prolific ones on a major publishing platform. We describe the creation of such a multidisciplinary corpus and highlight the obtained findings in terms of the following features: 1) a generic conceptual formalism for scientific entities in a multidisciplinary scientific context; 2) the feasibility of the domain-independent human annotation of scientific entities under such a generic formalism; 3) a performance benchmark obtainable for automatic extraction of multidisciplinary scientific entities using BERT-based neural models; 4) a delineated 3-step entity resolution procedure for human annotation of the scientific entities via encyclopedic entity linking and lexicographic word sense disambiguation; and 5) human evaluations of Babelfy returned encyclopedic links and lexicographic senses for our entities. Our findings cumulatively indicate that human annotation and automatic learning of multidisciplinary scientific concepts as well as their semantic disambiguation in a wide-ranging setting as STEM is reasonable. Keywords: Entity Recognition, Entity Classification, Entity Resolution, Entity Linking, Word Sense Disambiguation, Evaluation Corpus, Language Resource 1. Introduction Recently, an increasing number of research efforts are geared toward adopting Knowledge Graphs (KG) for mod- eling scholarly publications (Ammar et al., 2018; Jaradeh et al., 2019). They advocate for such advanced seman- tic machine-interpretability, via KGs, of the publications to enable their more intelligent automated processing (e.g., in search applications). To this end, KGs leveraged in aca- demic and industrial settings fostering better search already serve as case-in-point, albeit so far only over commonsense world knowledge (consider the DBpedia (Auer et al., 2007) and Google (Singhal, Amit, 2012) KGs as two prominent examples among others). To represent scholarly publications as KGs, from an Infor- mation Extraction (IE) perspective, and scientific IE, in par- ticular, extracting scientific entities from scholarly publica- tions becomes a vital task to address since entities lie at the core of KGs. While these scientific entities are essen- tially scientific terms, they are referred to with the broader notion of entities in the rest of the paper to posit them as valid knowledge graph node candidates. As an IE task, the extraction of scientific entities is being increasingly stud- ied in the Natural Language Processing (NLP) community– the SemEval series itself has so far seen four tasks orga- nized (Kim et al., 2010; Moro and Navigli, 2015; Augen- stein …

Content truncated to 500 words