Duluth at SemEval-2021 Task 11: Applying DeBERTa to Contributing Sentence Selection and Dependency Parsing for Entity Extraction

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 490–501 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 490 Duluth at SemEval-2021 Task 11: Applying DeBERTa to Contributing Sentence Selection and Dependency Parsing for Entity Extraction Anna Martin & Ted Pedersen Department of Computer Science University of Minnesota Duluth, MN 55812, USA {mart5877,tpederse}@d.umn.edu Abstract This paper describes the Duluth system that participated in SemEval-2021 Task 11, NLP Contribution Graph. It details the extraction of contribution sentences and scientific enti- ties and their relations from scholarly articles in the domain of Natural Language Process- ing. Our solution uses deBERTa for multi- class sentence classification to extract the con- tributing sentences and their type, and depen- dency parsing to extract phrases from each sen- tence and format into subject-predicate-object triples. Our system ranked fifth of seven for Phase 1: end-to-end pipeline, sixth of eight for Phase 2 Part 1: phrases and triples, and fifth of eight for Phase 2 Part 2: triples extraction. 1 Introduction The rapid rate at which scientific literature grows makes it difficult to keep up with new research even in one’s own field. Automated solutions are challenging since text formatted and written for human consumption does not lend itself to machine processing. (Jaradeh et al., 2019) have proposed a knowl- edge graph based system for collecting and struc- turing articles in a machine-readable format. This requires the annotation of many scholarly articles, a task that is time consuming to do by hand. The purpose of this SemEval task (D’Souza et al., 2021) is to enable the construction of a scholarly contribu- tions graph over English language NLP articles by automating the task of annotating scientific papers. This annotation processes consists of: 1. selecting sentences that describe the contri- bution of the article and outputting them to a sentences.txt file, 2. extracting scientific entities and relations from the selected sentences and outputting them to an entities.txt file, 3. and structuring the entities and relations into triples of the form subject-predicate-object. These triples are sorted into one of twelve in- formation units, which are labels that describe the type of contribution being made by the sentence outlined in the triples file. Following is an example of the annotation pro- cess using a sentence taken from the training data with the human–labeled entities in braces: We [apply] [dropout] of [0.4] [to layers], [0.3] [to RNN layers], [0.4] [to input em- bedding layers], [0.05] [to embedding layers], and [weight dropout] of [0.5] [to the RNN hidden-to-hidden matrix]. This task requires the 1) identification of this sentence as a contribution sentence belonging to the information unit HYPERPARAMETERS, 2) the extraction of the entities in braces, and 3) the for- matting of those entities into triples. This example and various others will be used throughout this paper to help contextualize our system description. SemEval-2021 Task 11 was organized into three phases, called Phase 1: end-to-end pipeline, Phase 2 Part 1: phrases and triples, and Phase 2 Part 2: triples …

Content truncated to 500 words