SUNNYNLP at SemEval-2018 Task 10: A Support-Vector-Machine-Based Method for Detecting Semantic Difference using Taxonomy and Word Embedding Features

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 741–746 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics SUNNYNLP at SemEval-2018 Task 10: A Support-Vector-Machine-Based Method for Detecting Semantic Difference using Taxonomy and Word Embedding Features Sunny Lai1,3, Kwong Sak Leung1,3 and Yee Leung2,3 1Department of Computer Science and Engineering 2Department of Geography and Resource Management 3Institute of Future Cities The Chinese University of Hong Kong Shatin, N.T., Hong Kong {slai, ksleung}@cse.cuhk.edu.hk, yeeleung@cuhk.edu.hk Abstract We present SUNNYNLP, our system for solv- ing SemEval 2018 Task 10: “Capturing Dis- criminative Attributes”. Our Support-Vector- Machine(SVM)-based system combines fea- tures extracted from pre-trained embeddings and statistical information from Is-A taxon- omy to detect semantic difference of concepts pairs. Our system is demonstrated to be ef- fective in detecting semantic difference and is ranked 1st in the competition in terms of F1 measure. The open source of our code is coined SUNNYNLP1. 1 Introduction Measuring semantic similarity between words has been a fundamental issue in Natural Language Processing (NLP). Semantic similarity measure- ments are used to improve downstream applica- tions including paraphrase detection (Xu et al., 2014), question answering (Lin, 2007), taxonomy enrichment (Jurgens and Pilehvar, 2016) and dia- logue state tracking (Mrksic et al., 2016). Despite the current success in using semantic model to measure semantic similarity, lesser at- tention is paid to teaching machines to make refer- ence (Searle, 1969; Abbott, 2010) to the real world in detecting semantic difference. The semantic difference detection problem can be formalized as a binary classification task: given a triplet (con- cept1, concept2, attribute) which comprises two concepts (e.g. apple, banana) and one attribute (e.g. red), determine whether the attribute charac- terizes the former concept but not the latter. Com- pared to pairwise semantic similarity detection, this problem is more complex than measuring sim- ilarity in general because of its underlying asym- metric property and the extra attribute involved. The SemEval 2018 Task 10 (Krebs et al., 2018) is 1https://github.com/Yermouth/sunnynlp therefore posed to attract attention to solving this problem. Although the task of semantic difference detec- tion is novel, similar tasks like referring expres- sion generation (REG) have been studied in the literature. Resources such as ontologies, knowl- edge bases (Krahmer and Van Deemter, 2012) and images (Kazemzadeh et al., 2014; Lazaridou et al., 2016) are used to learn referring expressions. The major difference between the present task and re- ferring expression is that REG systems can choose salient attributes for making successful reference to objects, while our system is required to decide whether a given attribute can be used to differenti- ate two similar objects. The rest of the paper is organized as follows: Section 2 explains our motivation and approach. Section 3 describes the official and external data used. Section 4 details our system implementa- tion. We analyze and discuss the result in Section 5 and conclude our work in Section 6. 2 General Approach Our approach to this problem is to divide the ternary concept-concept-attribute relationship (concept1, concept2, …

Content truncated to 500 words