Component-Enhanced Chinese Character Embeddings

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 829–834, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Component-Enhanced Chinese Character Embeddings Yanran Li1, Wenjie Li1, Fei Sun2, and Sujian Li3 1Department of Computing, The Hong Kong Polytechnic University, Hong Kong 2Institute of Computing Technology, Chinese Academy of Sciences, China 3Key Laboratory of Computational Linguistics, Peking University, MOE, China {csyli, cswjli}@comp.polyu.edu.hk, ofey.sunfei@gmail.com, lisujian@pku.edu.cn Abstract Distributed word representations are very useful for capturing semantic information and have been successfully applied in a variety of NLP tasks, especially on En- glish. In this work, we innovatively de- velop two component-enhanced Chinese character embedding models and their bi- gram extensions. Distinguished from En- glish word embeddings, our models ex- plore the compositions of Chinese char- acters, which often serve as semantic in- dictors inherently. The evaluations on both word similarity and text classification demonstrate the effectiveness of our mod- els. 1 Introduction Due to its advantage over traditional one-hot rep- resentation, distributed word representation has demonstrated its benefit for semantic representa- tion in various NLP tasks. Among the existing ap- proaches (Huang et al, 2012; Levy and Goldberg, 2014; Yang and Eisenstein, 2015), the continu- ous bag-of-words model (CBOW) and the continu- ous skip-gram model (SkipGram) remain the most popular ones that one can use to build word embed- dings efficiently (Mikolov et al, 2013a; Mikolov et al, 2013b). These two models learn the dis- tributed representation of a word based on its con- text. The context defined by the window of sur- rounding words may unavoidably include certain less semantically-relevant words and/or miss the words with important and relevant meanings (Levy and Goldberg, 2014). To overcome this shortcoming, a line of research deploys the order information of the words in the contexts by either deriving the contexts using de- pendency relations where the target word partici- pates (Levy and Goldberg, 2014; Yu and Dredze, 2014; Bansal et al, 2014) or directly keeping the order features (Ling et al, 2015). As to another line, Luong et al (2013) captures morphological composition by using neural networks and Qiu et al (2014) introduces the morphological knowl- edge as both additional input representation and auxiliary supervision to the neural network frame- work. While most previous work focuses on En- glish, there is a little work on Chinese. Zhang et al (2013) extracts the syntactical morphemes and Cheng et al (2014) incorporates the POS tags and dependency relations. Basically, the work in Chi- nese follows the same ideas as in English. Distinguished from English, Chinese characters are logograms, of which over 80% are phono- semantic compounds, with a semantic component giving a broad category of meaning and a phonetic component suggesting the sound1. For example, the semantic component 亻 (human) of the Chi- nese character 他 (he) provides the meaning con- nected with human. In fact, the components of most Chinese characters inherently bring with cer- tain levels of semantics regardless of the contexts. Being aware that the components of Chinese …

Content truncated to 500 words