TakeLab at SemEval-2018 Task 7: Combining Sparse and Dense Features for Relation Classification in Scientific Texts

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 842–847 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics TakeLab at SemEval-2018 Task 7: Combining Sparse and Dense Features for Relation Classification in Scientific Texts Martin Gluhak, Maria Pia di Buono, Abbas Akkasi, Jan ˇSnajder Text Analysis and Knowledge Engineering Lab Faculty of Electrical Engineering and Computing University of Zagreb Unska 3, 10000 Zagreb, Croatia {name.surname}@fer.hr Abstract We describe two systems for semantic relation classification with which we participated in the SemEval 2018 Task 7, subtask 1 on seman- tic relation classification: an SVM model and a CNN model. Both models combine dense pretrained word2vec features and hancrafted sparse features. For training the models, we combine the two datasets provided for the sub- tasks in order to balance the under-represented classes. The SVM model performed better than CNN, achieving an F1-macro score of 69.98% on subtask 1.1 and 75.69% on subtask 1.2. The system ranked 7th among 28 submis- sions on subtask 1.1 and 7th among 20 sub- missions on subtask 1.2. 1 Introduction Relation extraction is a traditional information ex- traction task which aims at detecting and classi- fying semantic relations between entities in text (Pawar et al., 2017). The task essentially induces structure from unstructured textual information, allowing us to obtain valuable information about the way in which entities interact, thus improv- ing human capacity to analyze (often large) quan- tities of textual data. Relation extraction is typi- cally framed as a classification task: pairs of en- tities from a document are inspected and the type of relation is predicted by means of local linguistic cues (Culotta et al., 2006). Relation extraction has been extensively stud- ied in the literature; see (Konstantinova, 2014) for a comprehensive overview. Etzioni et al. (2008) group the relation extraction approaches into three classes: (1) knowledge-based methods, (2) super- vised methods, and (3) self-supervised methods. Traditional approaches mostly relied on shallow machine learning models with handcrafted fea- tures (GuoDong et al., 2005) and specific kernel methods (Zelenko et al., 2003). Some systems leverage unlabeled data to improve classification and use semi-supervised or unsupervised learning (Chen et al., 2006; Hasegawa et al., 2004). The current state of the art is a deep recurrent neu- ral network model by Xu et al. (2016). Most re- search on relation extraction has leveraged stan- dard benchmark datasets from the ACE (Dodding- ton et al.) and SemEval-2010 Task 8 (Hendrickx et al., 2009). This paper describes the systems with which we participated in the SemEval 2018 task 7 on Semantic relation extraction and classification in scientific papers. We focused on the subtask 1 (relation classification), which featured two sce- narios: 1.1 Relation classification on clean (i.e., manually annotated) data and 1.2 Relation clas- sification on noisy (i.e., automatically annotated) data. Both scenarios start out with pre-extracted entity pairs, which makes the task simpler than re- lation extraction proper. On the other hand, the task remains challenging because of the choice of the domain: scientific …

Content truncated to 500 words