KnowGraph@IITK at SemEval-2021 Task 11: Building Knowledge Graph for NLP Research

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 467–477 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 467 KnowGraph@IITK at SemEval-2021 Task 11: Building Knowledge Graph for NLP Research Shashank Shailabh∗ Sajal Chaurasia∗ Ashutosh Modi Indian Institute of Technology Kanpur (IIT Kanpur) {shailabh, sajal}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract Research in Natural Language Processing is making rapid advances, resulting in the pub- lication of a large number of research papers. Finding relevant research papers and their con- tribution to the domain is a challenging prob- lem. In this paper, we address this challenge via the SemEval 2021 Task 11: NLPContri- butionGraph, by developing a system for a re- search paper contributions-focused knowledge graph over Natural Language Processing liter- ature. The task is divided into three sub-tasks: extracting contribution sentences that show im- portant contributions in the research article, extracting phrases from the contribution sen- tences, and predicting the information units in the research article together with triplet for- mation from the phrases. The proposed sys- tem is agnostic to the subject domain and can be applied for building a knowledge graph for any area. We found that transformer-based lan- guage models can significantly improve exist- ing techniques and utilized the SciBERT-based model. Our first sub-task uses Bidirectional LSTM (BiLSTM) stacked on top of SciBERT model layers, while the second sub-task uses Conditional Random Field (CRF) on top of SciBERT with BiLSTM. The third sub-task uses a combined SciBERT based neural ap- proach with heuristics for information unit pre- diction and triplet formation from the phrases. Our system achieved F1 score of 0.38, 0.63 and 0.76 in end-to-end pipeline testing, phrase extraction testing and triplet extraction testing respectively. 1 Introduction Given the advancements in Natural Language Pro- cessing (NLP), a large number of research papers are published every year. However, given the field’s dynamic nature, keeping track of all the papers is a non-trivial task. This motivated the formulation ∗ Authors equally contributed to this work. of an Open Research Knowledge Graph (Jaradeh et al., 2019), a knowledge graph of research contri- butions and the relation between them. Task 11 of SemEval 2021 (D’Souza et al., 2021) formalizes the building of a contributions-focused knowledge graph of NLP literature. The task is divided into three sub-tasks: • Sub-task A Extracting sentences that posit contributions in a research paper. • Sub-task B Extracting relevant phrases that include scientific terms and relational cues from the extracted sentences of the sub-task A. • Sub-task C Triplet (subject phrase, predicate phrase, object phrase) formation from the ex- tracted phrases of the sub-task B and clas- sification of the triplet in one of the infor- mation units (IU). There are twelve informa- tion units (Research problem, Approach, Re- sults, Model, Code, Dataset, Experimental setup, Hyperparameters, Baselines, Tasks, Ex- periments, and Ablation analysis), each fo- cusing on different sections in a research pa- per. These information units can represent all the findings given in a research paper. Out of twelve, first three information units are …

Content truncated to 500 words