Learning How to Translate North Korean through South Korean

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6711–6718 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6711 Learning How to Translate North Korean through South Korean Hwichan Kim†, Sangwhan Moon‡,∗, Naoaki Okazaki‡, Mamoru Komachi† †Tokyo Metropolitan University, 6-6 Asahigaoka, Hino, Tokyo 191-0065, Japan ‡Tokyo Institute of Technology, 2-12-1 Ookayama, Meguro, Tokyo 152-8550, Japan ∗Google LLC, 1600 Amphitheatre Parkway Mountain View, CA 94043, USA kim-hwichan@ed.tmu.ac.jp, sangwhan@iki.fi, okazaki@c.titech.ac.jp, komachi@tmu.ac.jp Abstract South and North Korea both use the Korean language, but there are some differences in their linguistic aspects, such as vocabulary and spelling rules. Korean NLP research has focused on South Korean only, and existing NLP systems for the Korean language, such as neural machine translation (NMT) models, cannot properly handle North Korean input. Training a model using North Korean data is the most straightforward approach to solving this problem, but there is insufficient data to train NMT models. In this study, we create data for North Korean NMT models using a comparable corpus. First, we manually create evaluation data for automatic alignment and machine translation. Then, we investigate automatic alignment methods suitable for North Korean data. Finally, we verify that a model trained using North Korean bilingual data without human annotation can significantly increase North Korean translation accuracy compared to existing South Korean models in zero-shot settings. Keywords: Parallel corpus construction, Machine translation, Korean 1. Introduction South and North Koreans both use the Korean lan- guage, with the same grammar; however, some linguis- tic aspects differ between the two (Lee, 1990; Yun and Kang, 2019). For example, there are both synonyms and homonyms between South and North variants of Korean language. Additionally, orthography differs, in- cluding word segmentation and an initial sound rule, which is a spelling rule unique to South Korean. Several NLP researchers have recently been working on the Korean language. For example, the Workshop of Asian Translation (Nakazawa et al., 2021) has been conducting a series of shared tasks annually, includ- ing those on Korean language variants. However, these studies have focused exclusively on the South Korean language; none of the developed NLP systems support the North Korean language. For example, public neu- ral machine translation (NMT) systems cannot trans- late North Korean-specific words (Table 1). Although training models on North Korean data is a simple and effective way to improve the quality of North Korean translation, parallel data for training are unavailable1. In this study, we tackle North Korean to English and Japanese bilingual data creation from comparable cor- pora to train a North Korean NMT model. Our contri- bution in this study is threefold: (1) We manually cre- ate North Korean evaluation data for the development of machine translation (MT) systems. (2) We investi- gate automatic article and sentence alignment methods 1Kim et al. (2020) proposed a North Korean and En- glish evaluation dataset for machine translation by manually rewriting sentences of a South Korean dataset to conform with North Korean spelling rules. However, as …

Content truncated to 500 words