Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3528–3533, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 3528 Don’t Just Scratch the Surface: Enhancing Word Representations for Korean with Hanja Kang Min Yoo∗, Taeuk Kim∗ and Sang-goo Lee Department of Computer Science and Engineering Seoul National University, Seoul, Korea {kangminyoo,taeuk,sglee}@europa.snu.ac.kr Abstract We propose a simple yet effective approach for improving Korean word representations using additional linguistic annotation (i.e. Hanja). We employ cross-lingual transfer learning in training word representations by leveraging the fact that Hanja is closely related to Chi- nese. We evaluate the intrinsic quality of rep- resentations learned through our approach us- ing the word analogy and similarity tests. In addition, we demonstrate their effectiveness on several downstream tasks, including a novel Korean news headline generation task. 1 Introduction There is a strong connection between the Korean and Chinese languages due to cultural and histori- cal reasons (Lee and Ramsey, 2011). Specifically, a set of logograms with very similar forms to the Chinese characters, called Hanja1, served in the past as the only medium for written Korean until Hangul, the Korean alphabet, and Jamo came into existence in 1443 (Sohn, 2001). Considering this etymological background, a substantial portion of Korean words are classified as Sino-Korean, a set of Korean words that had originated from Chinese and can be expressed in both Hanja and Hangul (Taylor, 1997), with the latter becoming common- place in modern Korean. Based on these facts, we assume the introduc- tion of Hanja-level information will aid in ground- ing better representation for the meaning of Ko- rean words (e.g. see Fig. 1). To validate our hypothesis, we propose a simple yet effective ap- proach to training Korean word representations, named Hanja-level SISG (Hanja-level Subword ∗Equal contribution. 1Hanja and traditional Chinese characters are very similar but not completely identical. Some differences in strokes and language-exclusive characters exist. Figure 1: An example of a Korean word showing its form and multi-level meanings. The Sino-Korean word consists of Hangul phonograms (KR) and Hanja lo- gograms (HJ). Although annotation of Hanja is op- tional, it offers deeper insight into the word meaning due to its association with the Chinese characters (CN). Information Skip-Gram), for capturing the seman- tics of Hanja and subword structures of Korean and introducing them into the vector space. Note that it is also quite intuitive for native Koreans to resolve the ambiguity of (Sino-)Korean words with the aid of Hanja. We conjecture this heuris- tic is attributed to the fact that Hanja characters are logograms, each of which contains more lex- ical meaning when compared against its counter- part, the Hangul character, which is a phonogram. Accordingly, in this work, we focus on exploiting the rich extra information from Hanja as a means of constructing better Korean word embeddings. Furthermore, our approach enables us to empir- ically investigate the potential of character-level cross-lingual knowledge transfer with a case …

Content truncated to 500 words