ECNUICA at SemEval-2021 Task 11: Rule based Information Extraction Pipeline

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1295–1302 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1295 ECNUICA at SemEval-2021 Task 11: Rule based Information Extraction Pipeline Jiaju Lin, Jing Ling ∗, Zhiwei Wang∗, Jiawei Liu∗, Qin Chen, Liang He School of Computer Science and Technology East China Normal University 51205901095@stu.ecnu.edu.cn Abstract This paper presents our endeavor for solving task11, NLPContributionGraph, of SemEval- 2021. The purpose of the task is to extract triples from a paper in the Nature Language Processing field for constructing an Open Re- search Knowledge Graph. The task includes three sub-tasks: detecting the contribution sen- tences in papers, identifying scientific terms and predicate phrases from the contribution sentences; and inferring triples in the form of (subject, predicate, object) as statements for Knowledge Graph building. In this paper, we apply an ensemble of various fine-tuned pre- trained language models (PLM) for tasks one and two. In addition, the self-training meth- ods are adopted for tackling the shortage of annotated data. For the third task, rather than using classic neural open information extrac- tion (OIE) architectures, we generate poten- tial triples via manually designed rules and de- velop a binary classifier to differentiate posi- tive ones from others. The quantitative results show that we obtain the 4th, 2nd, and 2nd rank in three evaluation phases. 1 Introduction The notion of Open Research Knowledge Graph (ORKG) is first proposed by (Jaradeh et al., 2019) who take steps toward a knowledge graph based infrastructure that acquires scholarly knowledge in machine actionable form. In that form, researchers can keep up with cutting edge academic achieve- ments and eliminate cognitive overload. To ac- celerate the construction of ORKG, an automatic system is expected. The SemEval-21 task11 is a triple extraction task targeted at building that system. As shown in Table1, the task is divided into three sub-parts corresponding to different pro- cessing steps: Sub-task A detects contribution sen- ∗equal contribution tences in English articles and classifies them into in- formation units such as Approaches, Models, and Ablation − analysis; Sub-task B extracts scien- tific terms and relational cue phrases from contribu- tion sentences; Sub-task C infers subject-predicate- object triples for KG building with the results of two previous sub-tasks. For Sub-task A—contribution sentence detec- tion—the evaluation data covers a larger sphere than the training data. Additionally, the amount of annotated samples differs among research fields. Hence, we use self-training to generate a set of silver samples for fields lacking gold data. An ensemble of fine-tuned PLM based classifiers is then deployed to categorize sentences. For sub- task B—scientific term extraction—we compared BERT based sequence labeling systems in detail and chose the best architecture. For sub-task C—triple generation—we give insight into the con- struction of triples and designed a rule for potential triples generation. A binary classifier is then ap- plied to distinguish the positive triples. Our quantitative results show data augmentation via self-training is of paramount importance for sub-task A. Although seldom is CRF used …

Content truncated to 500 words