Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6473 - 6484 December 7-11, 2022 ©2022 Association for Computational Linguistics Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modeling Zhijun Wang Xuebo Liu∗ Min Zhang Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China 1190303311@stu.hit.edu.cn, {liuxuebo,zhangmin2021}@hit.edu.cn Abstract Existing research generally treats Chinese char- acter as a minimum unit for representation. However, such Chinese character representa- tion will suffer two bottlenecks: 1) Learning bottleneck, the learning cannot benefit from its rich internal features (e.g., radicals and strokes); and 2) Parameter bottleneck, each individual character has to be represented by a unique vector. In this paper, we introduce a novel representation method for Chinese characters to break the bottlenecks, namely StrokeNet, which represents a Chinese char- acter by a Latinized stroke sequence (e.g., “凹(concave)” to “ajaie” and “凸(convex)” to “aeaqe”). Specifically, StrokeNet maps each stroke to a specific Latin character, thus allow- ing similar Chinese characters to have simi- lar Latin representations. With the introduc- tion of StrokeNet to neural machine transla- tion (NMT), many powerful but not applicable techniques to non-Latin languages (e.g., shared subword vocabulary learning and ciphertext- based data augmentation) can now be perfectly implemented. Experiments on the widely- used NIST Chinese-English, WMT17 Chinese- English and IWSLT17 Japanese-English NMT tasks show that StrokeNet can provide a signif- icant performance boost over the strong base- lines with fewer model parameters, achieving 26.5 BLEU on the WMT17 Chinese-English task which is better than any previously re- ported results without using monolingual data. Code and scripts are freely available at https: //github.com/zjwang21/StrokeNet. 1 Introduction Neural machine translation (NMT) has become the mainstream paradigm in machine translation re- cently (Sutskever et al., 2014; Cho et al., 2014; Bahdanau et al., 2015). With rich bilingual par- allel corpora, NMT achieves state-of-the-art per- formance on multiple translation benchmarks. In ∗Co-first and Corresponding Author Zh 布 什和沙龙举行了 会谈 En Bush held a talk with Sharon Zh (Stroke) etasa taea teatoaie oodatot etcto ootetneea ttaeer hr tneelo oyottoottn Table 1: StrokeNet represents a Chinese character by a Latinized stroke sequence. For example, “布” to “etasa” and “了” to “hr”. Chinese NMT tasks, the Chinese character has been the minimum representation unit for a long time. However, such representation perhaps might not be the best choice for Chinese NMT due to the two following representation bottlenecks. The first is the learning bottleneck. The repre- sentation learning of Chinese does not fully utilize its rich internal features. Latin languages have rich information in words like affixes. Actually, Chinese also has this kind of internal information. A Chinese character usually contains one radical (rarely has two) and several other radical-like com- ponents (Li et al., 2015). Characters with the rad- ical “扌” commonly are verbs. The characters “扎(tie)”, “拉(pull)”, “打(hit)”, “扔(throw)” and “提(carry)” all have the meaning of acting with hands because they have the same radical “扌”. Latin languages can easily learn this internal infor- mation by subword …

Content truncated to 500 words