Glyph2Vec: Learning Chinese Out-of-Vocabulary Word Embedding from Glyphs

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2865–2871 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2865 Glyph2Vec: Learning Chinese Out-of-Vocabulary Word Embedding from Glyphs Hong-You Chen∗ The Ohio State University chen.9301@osu.edu Sz-Han Yu∗ National Taiwan University r04922007@ntu.edu.tw Shou-De Lin National Taiwan University sdlin@csie.ntu.edu.tw Abstract Chinese NLP applications that rely on large text often contain huge amounts of vocabu- lary which are sparse in corpus. We show that characters’ written form, Glyphs, in ideo- graphic languages could carry rich semantics. We present a multi-modal model, Glyph2Vec, to tackle Chinese out-of-vocabulary word em- bedding problem. Glyph2Vec extracts visual features from word glyphs to expand current word embedding space for out-of-vocabulary word embedding, without the need of access- ing any corpus, which is useful for improv- ing Chinese NLP systems, especially for low- resource scenarios. Experiments across differ- ent applications show the significant effective- ness of our model. 1 Introduction Word embedding encoded semantic and syntac- tic information (Mikolov et al., 2013a,b) in low- dimensional space have served as useful features for various NLP applications but often require large-scale corpus with billions of tokens to train. A natural constraint of word embedding is that it is not practical to collect the entire vocabulary of any language with large enough frequency to train the embedding for every word, since some new words may appear in downstream tasks. A typical solution is to simply assign a specific UNK embedding to all out-of-vocabulary (OOV) words that do not appear in the training data. Current solutions such as using subwords (e.g., characters) are mainly considering alphabetic lan- guages (e.g., English and French) that are com- posed of small amount of characters. Such tech- niques may not be sufficient for ideographic lan- ∗ Equally contribution. Figure 1: Statistics of length of Chinese words in Sinica Corpus. guages (e.g., Chinese and Japanese) in which a word is often composed with characters of a large amounts. An example is that traditional Chinese includes about 17k distinct tokens. Therefore, it could be expected to suffer from underfitting not only word embedding but also character embed- ding. Even worse, words in ideographic languages are often composed of 2-3 characters only, unlike words in alphabetic languages are longer but with smaller types of characters. Figure 1 provides the statistics in Chinese Sinica Corpus. (a) (b) Figure 2: Example of compositionality of Chinese character components. (a) The same radical 火(fire) implies related meaning for 烤(roast) and 炸(fried). Components may also share similar semantics even they are different in graphs. 鳥 and 隹 are both re- fer to birds. (b) Cangjie input method. Each character can be presented as several keyboard inputs based on its components (e.g., 惆 is for 心+月+土+口). The visual structure (or glyph) of a Chinese character contains rich semantics. A Chinese char- http://asbc.iis.sinica.edu.tw/indexreadme.htm 2866 acter is made up of several graphical components. Figure 2 shows some examples that components in characters represent similar semantic or pronunci- ation. In addition to glyphs, we …

Content truncated to 500 words