Word Embedding-based Antonym Detection using Thesauri and Distributional Information

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 984–989, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Word Embedding-based Antonym Detection using Thesauri and Distributional Information Masataka Ono, Makoto Miwa, Yutaka Sasaki Department of Advanced Science and Technology Toyota Technological Institute 2-12-1 Hisakata, Tempaku-ku, Nagoya, Japan {sd12412, makoto-miwa, yutaka.sasaki}@toyota-ti.ac.jp Abstract This paper proposes a novel approach to train word embeddings to capture antonyms. Word embeddings have shown to capture synonyms and analogies. Such word embeddings, how- ever, cannot capture antonyms since they de- pend on the distributional hypothesis. Our approach utilizes supervised synonym and antonym information from thesauri, as well as distributional information from large-scale unlabelled text data. The evaluation results on the GRE antonym question task show that our model outperforms the state-of-the-art sys- tems and it can answer the antonym questions in the F-score of 89%. 1 Introduction Word embeddings have shown to capture synonyms and analogies (Mikolov et al., 2013b; Mnih and Kavukcuoglu, 2013; Pennington et al., 2014). Word embeddings have also been effectively employed in several tasks such as named entity recogni- tion (Turian et al., 2010; Guo et al., 2014), adjectival scales (Kim and de Marneffe, 2013) and text classi- fication (Le and Mikolov, 2014). Such embeddings trained based on distributional hypothesis (Harris, 1954), however, often fail to recognize antonyms since antonymous words, e.g. strong and weak, oc- cur in similar contexts. Recent studies focuses on learning word embeddings for specific tasks, such as sentiment analysis (Tang et al., 2014) and de- pendency parsing (Bansal et al., 2014; Chen et al., 2014). These motivate a new approach to learn word embeddings to capture antonyms. Recent studies on antonym detection have shown that thesauri information are useful in distinguishing antonyms from synonyms. The state-of-the-art sys- tems achieved over 80% in F-score on GRE antonym tests. Yih et al. (2012) proposed a Polarity Induc- ing Latent Semantic Analysis (PILSA) that incor- porated polarity information in two thesauri in con- structing a matrix for latent semantic analysis. They additionally used context vectors to cover the out-of- vocabulary words; however, they did not use word embeddings. Recently, Zhang et al. (2014) pro- posed a Bayesian Probabilistic Tensor Factorization (BPTF) model to combine thesauri information and existing word embeddings. They showed that the usefulness of word embeddings but they used pre- trained word embeddings. In this paper, we propose a novel approach to construct word embeddings that can capture antonyms. Unlike the previous approaches, our ap- proach directly trains word embeddings to represent antonyms. We propose two models: a Word Em- bedding on Thesauri information (WE-T) model and a Word Embeddings on Thesauri and Distributional information (WE-TD) model. The WE-T model re- ceives supervised information from synonym and antonym pairs in thesauri and infers the relations of the other word pairs in the thesauri from the su- pervised information. The WE-TD model incorpo- rates corpus-based contextual information (distribu- tional information) into the WE-T model, which en- ables …

Content truncated to 500 words