Break it Down into BTS: Basic, Tiniest Subword Units for Korean

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7007 - 7024 December 7-11, 2022 ©2022 Association for Computational Linguistics Break it Down into BTS: Basic, Tiniest Subword Units for Korean Nayeon Kim1∗, Jun-Hyung Park1∗, Joon-Young Choi2, Eojin Jeon2, Youjin Kang1, SangKeun Lee1,2 1Department of Computer Science and Engineering 2Department of Artificial Intelligence Korea University, Seoul, Republic of Korea {lilian1208, irish07, johnjames, skdlcm456, yjkang10, yalphy}@korea.ac.kr Abstract We introduce Basic, Tiniest Subword (BTS) units for the Korean language, which are in- spired by the invention principle of Hangeul, the Korean writing system. Instead of rely- ing on 51 Korean consonant and vowel let- ters, we form the letters from BTS units by adding strokes or combining them. To exam- ine the impact of BTS units on Korean lan- guage processing, we develop a novel BTS- based word embedding framework that is read- ily applicable to various models. Our experi- ments reveal that BTS units significantly im- prove the performance of Korean word em- bedding on all intrinsic and extrinsic tasks in our evaluation. In particular, BTS-based word embedding outperforms the state-of-the- art Korean word embedding by 11.8% in word analogy. We further investigate the unique ad- vantages provided by BTS units through in- depth analysis. Our code is available at https: //github.com/irishev/BTS. 1 Introduction Distributed word representations, known as word embeddings, play a fundamental role in natural lan- guage processing (NLP). With their capability to formulate semantic and syntactic relationships be- tween words, word embeddings serve as a useful feature for various NLP applications, such as ma- chine translation (Qi et al., 2018), sentence classifi- cation (Kim, 2014), and named entity recognition (Lample et al., 2016). Initial approaches to word embeddings, such as Word2vec (Mikolov et al., 2013a,b) and GloVe (Pennington et al., 2014), rep- resent each word in a vocabulary using distinct vectors. To improve the quality of word representa- tions, many approaches have considered the inter- nal structure of words with subword information. (Kim et al., 2016; Wieting et al., 2016; Bojanowski et al., 2017; Sasaki et al., 2019; Zhao et al., 2020). *Authors contributed equally. For non-English languages, prior studies (Park et al., 2018; Chen et al., 2020) have demonstrated that word embeddings should be designed to con- sider their linguistic structures. In particular, Ko- rean has a unique and complex linguistic structure as a morphologically rich and agglutinative lan- guage (Song, 2006). In the realm of Korean word embedding, decomposing a word into jamo letters1 is believed to be the best feasible option for captur- ing syntactic and semantic information of Korean words, which has been empirically supported by Park et al. (2018). However, previous approaches have overlooked the academic background of the Korean alpha- bet, Hangeul. According to Hunminjeongeum2, Hangeul involves five basic consonants (ㄱ, ㄴ, ㅁ, ㅅ, ㅇ) and three basic vowels ( · , ㅡ, ㅣ). Other letters are made from the basic letters by adding strokes or combining together (National Hangeul Museum, 2018). This unique invention principle of Hangeul …

Content truncated to 500 words