Bf3R at SemEval-2018 Task 7: Evaluating Two Relation Extraction Tools for Finding Semantic Relations in Biomedical Abstracts

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 816–820 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Bf3R at SemEval-2018 Task 7: Evaluating Two Relation Extraction Tools for Finding Semantic Relations in Biomedical Abstracts Mariana Neves1, Daniel Butzke1, Gilbert Sch¨onfelder1,2, Barbara Grune1 1 German Federal Institute for Risk Assessment (BfR) Diedersdorfer Weg 1, 12277, Berlin, Germany 2 Charit´e - Universit¨atsmedizin Berlin, Institute of Clinical Pharmacology and Toxicology, Charit´eplatz 1, 10117 Berlin, Germany mariana.lara-neves@bfr.bund.de Abstract Automatic extraction of semantic relations from text can support finding relevant infor- mation from scientific publications. We de- scribe our participation in Task 7 of SemEval- 2018 for which we experimented with two re- lations extraction tools - jSRE and TEES - for the extraction and classification of six re- lation types. The results we obtained with TEES were significantly superior than those with jSRE (33.4% vs. 30.09% and 20.3% vs. 16%). Additionally, we utilized the model trained with TEES for extracting semantic re- lations from biomedical abstracts, for which we present a preliminary evaluation. 1 Introduction Finding relevant publications for a certain topic is an important task daily carried out by most re- searchers in various domains, such as computer science or biomedicine. However, most informa- tion retrieval methods usually consider only words and terms (named entities) and do not usually profit from semantic relationships between these entities (Lu, 2011). Many approaches frequently consider words and entities as bags of words but do not take advantage from intrinsic properties of scientific texts, such as subsections (e.g., introduc- tion, methods, results), common concepts (e.g., task, material) and relations between these con- cepts (e.g., model-feature, part-whole). However, extracting semantic relations from scientific text can potentially support finding relevant informa- tion for a certain topic by focusing on particular terms which participate in those relations. In ad- dition, the relation type and the corresponding ar- guments provide further information regarding the role that a certain entity plays in the text. We describe the experiments that we car- ried out during our participation in Subtask 2 of SemEval-2018 Task 71 (G´abor et al., 2018). The task consisted on the extraction of six semantic relations from scientific abstracts, namely: “USAGE”, “RESULT”, “MODEL”, “PART WHOLE”, “TOPIC” and “COMPARI- SON”. While the entities were given (and all be- long to the general type “ENTITY”), participants of subtask 2 were required to identify the rela- tions and classify these into one of the six types. All relations were asymmetrical (regarding their direction), except for “COMPARISON”, and the identification of the direction of the relations was mandatory. The documents came from the ACL Anthology2, thus belonged to the domain of com- putational linguistic, and were derived from a more comprehensive corpus which includes more relations than the ones under evaluation in the challenge (G´abor et al., 2016). Our contribution in this work is two-fold: (a) we experimented with two available relation ex- traction (RE) tools in the context of the Subtask 2 of SemEval-2018 Task 7; and (b) …

Content truncated to 500 words