Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 76–85, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network Kim Anh Nguyen and Sabine Schulte im Walde and Ngoc Thang Vu Institut f¨ur Maschinelle Sprachverarbeitung Universit¨at Stuttgart Pfaffenwaldring 5B, 70569 Stuttgart, Germany {nguyenkh,schulte,thangvu}@ims.uni-stuttgart.de Abstract Distinguishing between antonyms and synonyms is a key task to achieve high performance in NLP systems. While they are notoriously difficult to distinguish by distributional co-occurrence models, pattern-based methods have proven effec- tive to differentiate between the relations. In this paper, we present a novel neu- ral network model AntSynNET that ex- ploits lexico-syntactic patterns from syn- tactic parse trees. In addition to the lexi- cal and syntactic information, we success- fully integrate the distance between the re- lated words along the syntactic path as a new pattern feature. The results from clas- sification experiments show that AntSyn- NET improves the performance over prior pattern-based methods. 1 Introduction Antonymy and synonymy represent lexical se- mantic relations that are central to the organization of the mental lexicon (Miller and Fellbaum, 1991). While antonymy is defined as the oppositeness be- tween words, synonymy refers to words that are similar in meaning (Deese, 1965; Lyons, 1977). From a computational point of view, distinguish- ing between antonymy and synonymy is impor- tant for NLP applications such as Machine Trans- lation and Textual Entailment, which go beyond a general notion of semantic relatedness and require to identify specific semantic relations. However, due to interchangeable substitution, antonyms and synonyms often occur in similar contexts, which makes it challenging to automatically distinguish between them. Two families of approaches to differentiate be- tween antonyms and synonyms are predominent in NLP. Both make use of distributional vector representations, relying on the distributional hy- pothesis (Harris, 1954; Firth, 1957), that words with similar distributions have related meanings: co-occurrence models and pattern-based models. These distributional semantic models (DSMs) of- fer a means to represent meaning vectors of words or word pairs, and to determine their semantic re- latedness (Turney and Pantel, 2010). In co-occurrence models, each word is repre- sented by a weighted feature vector, where fea- tures typically correspond to words that co-occur in particular contexts. When using word embed- dings, these models rely on neural methods to rep- resent words as low-dimensional vectors. To cre- ate the word embeddings, the models either make use of neural-based techniques, such as the skip- gram model (Mikolov et al., 2013), or use matrix factorization (Pennington et al., 2014) that builds word embeddings by factorizing word-context co- occurrence matrices. In comparison to standard co-occurrence vector representations, word em- beddings address the problematic sparsity of word vectors and have achieved impressive results in many NLP tasks such as word similarity (e.g., Pennington et al. (2014)), relation classification (e.g., Vu et al. (2016)), and antonym-synonym dis- tinction (e.g., Nguyen et al. (2016)). In pattern-based models, vector representations make use of …

Content truncated to 500 words