Multi-Granularity Chinese Word Embedding

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 981–986, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Multi-Granularity Chinese Word Embedding Rongchao Yin†‡, Quan Wang†‡ , Rui Li†‡, Peng Li†‡∗, Bin Wang†‡ †Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China ‡University of Chinese Academy of Sciences, Beijing 100049, China {yinrongchao,wangquan,lirui,lipeng,wangbin}@iie.ac.cn Abstract This paper considers the problem of learning Chinese word embeddings. In contrast to En- glish, a Chinese word is usually composed of characters, and most of the characters them- selves can be further divided into components such as radicals. While characters and radical- s contain rich information and are capable of indicating semantic meanings of words, they have not been fully exploited by existing word embedding methods. In this work, we propose multi-granularity embedding (MGE) for Chi- nese words. The key idea is to make full use of such word-character-radical composition, and enrich word embeddings by further incorpo- rating finer-grained semantics from characters and radicals. Quantitative evaluation demon- strates the superiority of MGE in word sim- ilarity computation and analogical reasoning. Qualitative analysis further shows its capabili- ty to identify finer-grained semantic meanings of words. 1 Introduction Word embedding, also known as distributed word representation, is to represent each word as a real- valued low-dimensional vector, through which the semantic meaning of the word can be encoded. Re- cent years have witnessed tremendous success of word embedding in various NLP tasks (Bengio et al., 2006; Mnih and Hinton, 2009; Collobert et al., 2011; Zou et al., 2013; Kim, 2014; Liu et al., 2015; Iyyer et al., 2015). The basic idea behind is to learn the distributed representation of a word using its con- text. Among existing approaches, the continuous bag-of-words model (CBOW) and Skip-Gram mod- el are simple and effective, capable of learning word embeddings efficiently from large-scale text corpo- ra (Mikolov et al., 2013a; Mikolov et al., 2013b). ∗Corresponding author: Peng Li. Besides the success in English, word embedding has also been demonstrated to be extremely useful for Chinese language processing (Xu et al., 2015; Yu et al., 2015; Zhou et al., 2015; Zou et al., 2013). The work on Chinese generally follows the same idea as on English, i.e., to learn the embedding of a word on the basis of its context. However, in contrast to English where words are usually taken as basic semantic units, Chinese words may have a complicated composition structure of their seman- tic meanings. More specifically, a Chinese word is often composed of several characters, and most of the characters themselves can be further divided in- to components such as radicals (部首).1 Both char- acters and radicals may suggest the semantic mean- ing of a word, regardless of its context. For exam- ple, the Chinese word “吃饭 (have a meal)” con- sists of two characters “ 吃 (eat)” and “饭 (meal)”, where “ 吃 (eat)” has the radical of “口 (mouth)”, and “饭 (meal)” the radical of “ 饣 (food)”. The se- …

Content truncated to 500 words