Learning Chinese Word Representations From Glyphs Of Characters

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 264–273 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Learning Chinese Word Representations From Glyphs Of Characters Tzu-Ray Su and Hung-Yi Lee Dept. of Electrical Engineering, National Taiwan University No. 1, Sec. 4, Roosevelt Road, Taipei, Taiwan {b01901007,hungyilee}@ntu.edu.tw Abstract In this paper, we propose new methods to learn Chinese word representations. Chi- nese characters are composed of graphical components, which carry rich semantics. It is common for a Chinese learner to com- prehend the meaning of a word from these graphical components. As a result, we propose models that enhance word repre- sentations by character glyphs. The char- acter glyph features are directly learned from the bitmaps of characters by con- volutional auto-encoder(convAE), and the glyph features improve Chinese word rep- resentations which are already enhanced by character embeddings. Another contri- bution in this paper is that we created sev- eral evaluation datasets in traditional Chi- nese and made them public. 1 Introduction No matter which target language it is, high quality word representations (also known as word “em- beddings”) are keys to many natural language processing tasks, for example, sentence classi- fication (Kim, 2014), question answering (Zhou et al., 2015), machine translation (Sutskever et al., 2014), etc. Besides, word-level representations are building blocks in producing phrase-level (Cho et al., 2014) and sentence-level (Kiros et al., 2015) representations. In this paper, we focus on learning Chinese word representations. A Chinese word is com- posed of characters which contain rich seman- tics. The meaning of a Chinese word is often re- lated to the meaning of its compositional charac- ters. Therefore, Chinese word embedding can be enhanced by its compositional character embed- dings (Chen et al., 2015; Xu et al., 2016). Further- more, a Chinese character is composed of several graphical components. Characters with the same component share similar semantic or pronuncia- tion. When a Chinese user encounters a previ- ously unseen character, it is instinctive to guess the meaning (and pronunciation) from its graph- ical components, so understanding the graphical components and associating them with semantics help people learning Chinese. Radicals1 are the graphical components used to index Chinese char- acters in a dictionary. By identifying the radical of a character, one obtains a rough meaning of that character, so it is used in learning Chinese word embedding (Yin et al., 2016) and character embed- ding (Sun et al., 2014; Li et al., 2015). However, other components in addition to radicals may con- tain potentially useful information in word repre- sentation learning. Our research begins with a question: Can ma- chines learn Chinese word representations from glyphs of characters? By exploiting the glyphs of characters as images in word representation learn- ing, all the graphical components in a character are considered, not limited to radicals. In our proposed methods, we render character glyphs to fixed-size grayscale images which are referred to as “character bitmaps”, as illustrated in Fig.1. A similar idea was also used in …

Content truncated to 500 words