YNU-HPCC at SemEval-2021 Task 11: Using a BERT Model to Extract Contributions from NLP Scholarly Articles

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 478–484 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 478 YNU-HPCC at SemEval-2021 Task 11: Using a BERT Model to Extract Contributions from NLP Scholarly Articles Xinge Ma, Jin Wang and Xuejie Zhang School of Information Science and Engineering Yunnan University Kunming, China Contact: maxinge@mail.ynu.edu.cn, {wangjin, xjzhang}@ynu.edu.cn Abstract This paper describes the system we built as the YNU-HPCC team in the SemEval-2021 Task 11: NLPContributionGraph. This task involves first identifying sentences in the giv- en natural language processing (NLP) schol- arly articles that reflect research contribution- s through binary classification; then identify- ing the core scientific terms and their relation phrases from these contribution sentences by sequence labeling; and finally, these scientific terms and relation phrases are categorized, i- dentified, and organized into subject-predicate- object triples to form a knowledge graph with the help of multiclass classification and multi- label classification. We developed a system for this task using a pre-trained language represen- tation model called BERT that stands for Bidi- rectional Encoder Representations from Trans- formers, and achieved good results. The av- erage F1-score for Evaluation Phase 2, Part 1 was 0.4562 and ranked 7th, and the average F1-score for Evaluation Phase 2, Part 2 was 0.6541, and also ranked 7th. 1 Introduction As the number of research publications increases, there is a growing need for digital libraries to equip researchers with alternative knowledge represen- tations. In addition, because scientific literature is growing at a rapid rate and researchers today are faced with a publication deluge, it is difficult to keep up with the research progress even within ones own narrow discipline. The open research knowledge graph (ORKG) (Jaradeh et al., 2019) is posited as a solution to the problem of keeping track of research progress without the cognitive overload imposed by reading dozens of full paper- s. To this end, the aim of this task is to build a comprehensive knowledge graph that represents the research contributions of scholarly publications per paper and also shows where the contribution- s are interconnected across papers (D’Souza and Auer, 2020). The task was defined on a dataset containing nat- ural language processing (NLP) scholarly articles with their contributions structured to be integrable within a knowledge graph infrastructure, such as the ORKG. The structured contribution annotation- s were provided as follows: (1) contribution sen- tences: a set of sentences about the contribution in the article; (2) scientific terms and relations: a set of scientific terms and relational cue phras- es extracted from the contribution sentences; and (3) triples: semantic statements that pair scientific terms with a relation, modeled toward the subject- predicate-object statements for building knowledge graph. The triples were organized under three (mandatory) or more of the twelve total informa- tion units (i.e., ResearchProblem, Approach, Mod- el, Code, Dataset, ExperimentalSetup, Hyperpa- rameters, Baselines, Results, Tasks, Experiments, and AblationAnalysis). An illustration of this pro- cess is shown in Figure 1. The difficulty of this task lies in text …

Content truncated to 500 words