Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4651–4658 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4651 Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results Ian H. Magnusson♠♥ and Scott E. Friedman♠ ♠SIFT, Minneapolis, MN, USA ♥Northeastern University, Boston, MA, USA magnusson.i@northeastern.edu friedman@sift.net Abstract Recent transformer-based approaches demon- strate promising results on relational scientific information extraction. Existing datasets fo- cus on high-level description of how research is carried out. Instead we focus on the sub- tleties of how experimental associations are presented by building SciClaim, a dataset of scientific claims drawn from Social and Be- havior Science (SBS), PubMed, and CORD- 19 papers. Our novel graph annotation schema incorporates not only coarse-grained entity spans as nodes and relations as edges between them, but also fine-grained attributes that mod- ify entities and their relations, for a total of 12,738 labels in the corpus. By including more label types and more than twice the la- bel density of previous datasets, SciClaim cap- tures causal, comparative, predictive, statis- tical, and proportional associations over ex- perimental variables along with their qualifi- cations, subtypes, and evidence. We extend work in transformer-based joint entity and rela- tion extraction to effectively infer our schema, showing the promise of fine-grained knowl- edge graphs in scientific claims and beyond. 1 Introduction Using relations as edges to connect nodes consist- ing of extracted entity mention spans produces ex- pressive and unambiguous knowledge graphs from unstructured text. This approach has been applied to diverse domains from moral reasoning in so- cial media (Friedman et al., 2021b) to qualitative structure in ethnographic texts (Friedman et al., 2021a), and is particularly useful for reasoning about scientific claims, where several experimental variables in a sentence may have differing rela- tions. Scientific information extraction datasets such as SciERC (Luan et al., 2018) use relations for labeling general scientific language. Utilizing the advances of SciBERT (Beltagy et al., 2019) in scientific language modeling, SpERT (Eberts and Text: Levels of social support for medical staff were significantly associated with self - efficacy and sleep quality and negatively associated with the degree of anxiety and stres Levels of social support Factor self - efficacy Factor q+ sleep quality Factor q+ the degree of anxiety Factor q- stress Factor q- for medical staff Qualifier significantly Magnitude associated with Association (Correlation, Sign+) arg0 arg1 arg1 negatively Magnitude associated with Association (Correlation, Sign-) arg0 arg1 arg1 Input: “Levels of social support for medical staff were significantly associated with self-efficacy and sleep quality and negatively associated with the degree of anxiety and stress.” Figure 1: SciClaim knowledge graph with entities (nodes), relations (edges), and attributes (parentheti- cals) connecting an independent variable via arg0 to distinct correlations with dependent variables via arg1. Ulges, 2020)—a transformer-based joint entity and relation extraction model—advanced the state of the art on SciERC. To extend relational scientific information ex- traction to specifically target scientific claims, we annotate SciClaim,1 a dataset of 12,738 anno- tations on 901 sentences from expert …

Content truncated to 500 words