ClaiRE at SemEval-2018 Task 7: Classification of Relations using Embeddings

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 836–841 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics ClaiRE at SemEval-2018 Task 7: Classification of Relations using Embeddings Lena Hettinger, Alexander Dallmann, Albin Zehe, Thomas Niebler and Andreas Hotho DMIR Group University of Wuerzburg {hettinger,dallmann,zehe,niebler,hotho} @informatik.uni-wuerzburg.de Abstract In this paper we describe our system for SemEval-2018 Task 7 on classification of se- mantic relations in scientific literature for clean (subtask 1.1) and noisy data (subtask 1.2). We compare two models for classifica- tion, a C-LSTM which utilizes only word em- beddings and an SVM that also takes hand- crafted features into account. To adapt to the domain of science we train word embeddings on scientific papers collected from arXiv.org. The hand-crafted features consist of lexical features to model the semantic relations as well as the entities between which the rela- tion holds. Classification of Relations using Embeddings (ClaiRE) achieved an F1 score of 74.89 % for the first subtask and 78.39 % for the second. 1 Introduction The goal of SemEval-2018 Task 7 is to extract and classify semantic relations between entities into six categories that are specific to scientific liter- ature (G´abor et al., 2018). In this work, we fo- cus on the subtask of classifying relations between entities in manually (subtask 1.1) and automati- cally annotated and therefore noisy data (subtask 1.2). Given a pair of related entities, the task is to classify the type of their relation among the fol- lowing options: Compare, Model-Feature, Part Whole, Result, Topic or Usage. Re- lation types are explained in detail in the task de- scription paper (G´abor et al., 2018). The follow- ing sentence shows an example of a Result rela- tion between the two entities combination meth- ods and system performance: Combination methods are an effec- tive way of improving system perfor- mance. This sentence is a good example for two chal- lenges we face in this task. First, almost half of all entities consist of noun phrases which has to be considered when constructing features. Secondly, the vocabulary is domain dependent and therefore background knowledge should be adopted. Previous approaches for semantic relation clas- sification tasks mainly employed two strategies. Either they made use of a lot of hand-crafted fea- tures or they utilized a neural network with as few background knowledge as possible. The winning system of an earlier SemEval challenge on relation classification (Hendrickx et al., 2009) adopted the first approach and achieved an F1 score of 82.2% (Rink and Harabagiu, 2010). Later, other works outperformed this approach by using CNNs with and without hand-crafted features (Santos et al., 2015; Xu et al., 2015) as well as RNNs (Miwa and Bansal, 2016). Approach We present two approaches that use different levels of preliminary information. Our first approach is inspired by the winning method of the SemEval-2010 challenge (Rink and Harabagiu, 2010). It models semantic relations by describing the two entities, between which the semantic relation holds, as well as the words …

Content truncated to 500 words