ETH-DS3Lab at SemEval-2018 Task 7: Effectively Combining Recurrent and Convolutional Neural Networks for Relation Classification and Extraction

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 689–696 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics ETH-DS3Lab at SemEval-2018 Task 7: Effectively Combining Recurrent and Convolutional Neural Networks for Relation Classification and Extraction Jonathan Rotsztejn1, Nora Hollenstein1,2, Ce Zhang1 1 Systems Group, ETH Zurich {rotsztej,noraho}@ethz.ch, ce.zhang@inf.ethz.ch 2 IBM Research, Zurich Abstract Reliably detecting relevant relations between entities in unstructured text is a valuable re- source for knowledge extraction, which is why it has awaken significant interest in the field of Natural Language Processing. In this paper, we present a system for relation classification and extraction based on an ensemble of con- volutional and recurrent neural networks that ranked first in 3 out of the 4 subtasks at Se- mEval 2018 Task 7. We provide detailed ex- planations and grounds for the design choices behind the most relevant features and analyze their importance. 1 Introduction and related work One of the current challenges in analyzing un- structured data is to extract valuable knowledge by detecting the relevant entities and relations be- tween them. The focus of SemEval 2018 Task 7 is on relation classification (assigning a type of re- lation to an entity pair - Subtask 1) and relation extraction (detecting the existence of a relation be- tween two entities and determining its type - Sub- task 2). Moreover, the task distinguishes between rela- tion classification on clean data (i.e.: manually annotated entities - Subtask 1.1) and noisy data (automatically annotated entities - Subtask 1.2). It addresses semantic relations from 6 categories, all of them specific to scientific literature. Rela- tion instances are to be classified into one of the following classes: USAGE, RESULT, MODEL- FEATURE, PART-WHOLE, TOPIC, COMPARE, where the first five are asymmetrical relations and the last is order-independent (see G´abor et al. (2018) for a more detailed description of the task). Since the training data was provided by the task organizers, we focused on supervised methods for relation classification and extraction. Similar sys- tems in the past have been based on Support Vec- tor Machines (Uzuner et al., 2011; Minard et al., 60 65 70 75 80 ICC CSD OP RS WCE EN RPE PTE CRC GD WNC %"F1"score Added"feature Figure 1: Feature addition study to evaluate the impact of the most relevant features on the F1 score of the 5- fold cross-validated training set of Subtasks 1.1 and 1.2 2011), Na¨ıve Bayes (Zayaraz et al., 2015) and Conditional Random Fields (Sutton and McCal- lum, 2006). More recent approaches have experi- mented with neural network architectures (Socher et al., 2012; Fu et al., 2017), especially convolu- tional neural networks (CNNs) (Nguyen and Gr- ishman, 2015; Lee et al., 2017) and recurrent neu- ral networks (RNNs) based on LSTMs (Zheng et al., 2017; Peng et al., 2017). The system pre- sented in this article builds upon the latest im- provements in employing neural networks for re- lation classification and extraction. An overview of the most relevant features is shown on Figure 1. …

Content truncated to 500 words