Talla at SemEval-2018 Task 7: Hybrid Loss Optimization for Relation Classification using Convolutional Neural Networks

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 863–867 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics Talla at SemEval-2018 Task 7: Hybrid Loss Optimization for Relation Classification using Convolutional Neural Networks Bhanu Pratap, Daniel Shank, Oladipo Ositelu, Byron V. Galbraith Talla Inc. Boston, MA, USA {bhanu, daniel, ladi, byron}@talla.com Abstract This paper describes our approach SemEval-2018 Task 7 – given an entity-to tagged text from the ACL Anthology corpus, identify and classify pairs of entities that have one of six possible semantic relationships. Our model consists of a convolutional neural network leveraging pre-trained word embed- dings, unlabeled ACL-abstracts, and multiple window sizes to automatically learn useful features from entity-tagged sentences. We also experiment with a hybrid loss function, a combination of cross-entropy loss and ranking loss, to boost the separation in classification scores. Lastly, we include WordNet-based features to further improve the performance of our model. Our best model achieves an F1(macro) score of 74.2 and 84.8 on subtasks 1.1 and 1.2, respectively. 1 Introduction Classifying the relationship between entities is an important natural language processing (NLP) task that serves as a building block for a variety of NLP applications such as knowledge base con- struction and question-answering tasks. SemEval- 2018 Task 7 (G´abor et al., 2018) provided entity- tagged texts from the ACL Anthology corpus and asked participants to identify and classify entity pairs into one of six semantic relationships. Our approach to this problem consisted of se- lecting two architectures shown to be successful (Zeng et al., 2014; Nguyen and Grishman, 2015) in this problem domain and adapting them to this particular task. We found that pre-trained word embeddings were effective for this problem, as well as a combined loss function, and using Word- Net features at a later stage of our model. Figure 1: An example of sentence-level relation In- stance. Sentences marked with entity positions were the input to our model. 2 System Description Convolutional neural networks (CNNs) have been proven to significantly outperform other methods for Relation Classification (Zeng et al., 2014; dos Santos et al., 2015; Nguyen and Grishman, 2015). Our approach was inspired by Nguyen and Grish- man (2015) and dos Santos et al. (2015), both sys- tems being minimally dependent on explicit fea- ture engineering. While Nguyen and Grishman (2015) relied solely on their model architecture to automatically extract useful features, we also in- cluded additional features based on part-of-speech tags and WordNet hypernyms. Following dos San- tos et al. (2015), we trained our model on a hybrid objective function, a combination of cross entropy loss and ranking loss. Finally, we also trained our model in two stages to utilize large amounts of un- labeled ACL corpus abstracts (Bird et al., 2008). We describe these stages in detail in section 4.1. Each abstract is first tokenized into sentences. For each sentence, we then formed training exam- ples by taking all combinations of pairs of enti- ties annotated in the sentence. If a pair is anno- …

Content truncated to 500 words