Syntactic Dependencies and Distributed Word Representations for Analogy Detection and Mining

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2441–2450, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Syntactic Dependencies and Distributed Word Representations for Chinese Analogy Detection and Mining Likun Qiu1,2, Yue Zhang2, Yanan Lu3 1School of Chinese Language and Literature, Ludong University, China 2Singapore University of Technology and Design, Singapore 3Computer School, Wuhan University, China qiulikun@pku.edu.cn, yue zhang@sutd.edu.sg, luyanan@whu.edu.cn Abstract Distributed word representations capture relational similarities by means of vec- tor arithmetics, giving high accuracies on analogy detection. We empirically inves- tigate the use of syntactic dependencies on improving Chinese analogy detection based on distributed word representation- s, showing that a dependency-based em- beddings does not perform better than an ngram-based embeddings, but dependen- cy structures can be used to improve anal- ogy detection by filtering candidates. In addition, we show that distributed repre- sentations of dependency structure can be used for measuring relational similarities, thereby help analogy mining. 1 Introduction Relational similarity measures the correspondence between word-word relations (Medin et al., 1990). It is relevant to many tasks in NLP (Turney, 2006), such as word sense disambiguation, information extraction, question answering, information re- trieval, semantic role identification and metaphor detection. Typical tasks on relational similarity in- clude analogy detection, which measures the de- gree of relational similarities, and analogy mining, which extracts analogous word pairs from unstruc- tured text. Recently, distributed word representations (i.e. embeddings) (Mikolov et al., 2013a; Mikolov et al., 2013b; Levy and Goldberg, 2014b) have been used for unsupervised analogy detection. Mikolov et al. use attributional similarities between words in a relation to compute relational similarities, and show that the method outperforms the best sys- tem in the SemEval 2012 shared task on analo- gy detection. Levy and Goldberg (2014b) fur- ther improve Mikolov’s relational similarity mea- sure method using novel arithmetic combination- s of attributional similarities. For simplicity, we call the method of Mikolov et al. embedding- based analogy detection, without stressing the dif- ference between distributed and distributional (i.e. counting-based) word representations. Most work on embedding-based analogy detec- tion uses relational similarities as a measure of the quality of embeddings. However, relatively little has been done in the opposite direction, exploring how to leverage embeddings for improving rela- tional similarity algorithms. We empirically study the use of word embeddings for Chinese analogy detection and mining, leveraging syntactic depen- dencies, which has been shown to be closely asso- ciated with semantic relations (Levin, 1993; Chi- u et al., 2007). Compared with many other lan- guages, this association is particularly strong for Chinese, which is fully configurational and lack- s morphology. To our knowledge, relatively little work has been reported on Chinese relational sim- ilarities, compared to other tasks in Chinese NLP, including syntactic parsing, information extraction and machine translation. We work on three specific problems. First, we study the effect of dependency-based word em- beddings for analogy detection. There are two variations of Mikolov et al’s skip-gram embed- ding model, one training the distributed word rep- resentation …

Content truncated to 500 words