Unraveling Antonym’s Word Vectors through a Siamese-like Network

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3297–3307 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3297 Unraveling Antonym’s Word Vectors through a Siamese-like Network Mathias Etcheverry Republic University Montevideo, Uruguay mathiase@fing.edu.uy Dina Wonsever Republic University Montevideo, Uruguay wonsever@fing.edu.uy Abstract Discriminating antonyms and synonyms is an important NLP task that has the difficulty that both, antonyms and synonyms, contains sim- ilar distributional information. Consequently, pairs of antonyms and synonyms may have similar word vectors. We present an ap- proach to unravel antonymy and synonymy from word vectors based on a siamese network inspired approach. The model consists of a two-phase training of the same base network: a pre-training phase according to a siamese model supervised by synonyms and a train- ing phase on antonyms through a siamese-like model that supports the antitransitivity present in antonymy. The approach makes use of the claim that the antonyms in common of a word tend to be synonyms. We show that our ap- proach outperforms distributional and pattern- based approaches, relaying on a simple feed forward network as base network of the train- ing phases. 1 Introduction Antonymy and synonymy are lexical relations that are crucial in language semantics. Antonymy is the relation between opposite words, (e.g. big- small) and synonymy refers to words with similar meaning (e.g. bug-insect). Detecting them auto- matically is a challenging NLP task that can ben- efit many others like textual entailment (Haghighi et al., 2005; Snow et al., 2006), machine transla- tion (Bar and Dershowitz, 2010) and abstractive summarization (Khatri et al., 2018). Hand crafted lexical databases, such as Word- Net (Miller, 1995), have been built and main- tained to be used in NLP and other fields con- taining antonyms, synonyms and other lexical se- mantic relations. However, its construction and maintenance takes a considerable human effort and it is difficult to achieve a broad coverage. De- tecting antonyms automatically, relying on exis- tent resources such as text, dictionaries and lexical databases is an active NLP research area. In the last decade, the use and research concern- ing word vectors have increased rapidly. Word vectors rely on words co-occurrence information in a large corpus. The key idea behind word vec- tors is the distributional hypothesis that can be ex- pressed as ”the words that are similar in meaning tend to occur in similar contexts” (Sahlgren, 2008; Rubenstein and Goodenough, 1965). A variety of methods have been developed to train word vec- tors, such as skip-gram (Mikolov et al., 2013), GloVe (Pennington et al., 2014), FastText (Joulin et al., 2016) and ElMo (Peters et al., 2018). Word vectors are used widely in NLP, for example, a well-known use is in supervised learning, taking advantage of the expansion through words relat- edness of the training data. A main problem to discriminate antonymy au- tomatically in a distributional unsupervised set- ting is that the oppositeness is not easily distin- guishable in terms of the context distributions. In fact, pairs of …

Content truncated to 500 words