Chinese Spelling Check based on Neural Machine Translation

Chinese Spelling Check based on Neural Machine Translation Chiao-Wen Li1, Jhih-Jie Chen2, Jason S. Chang2 1Institute of Information Systems and Applications National Tsing Hua University 2Department of Computer Science National Tsing Hua University {chiaowen, jjc, jason}@nlplab.cc Abstract This paper presents a method for Chinese spelling check that automatically learns to cor- rect a sentence with potential spelling errors. In our approach, a character-based neural ma- chine translation (NMT) model is trained to translate a potentially misspelled sentence into correct one, using right-and-wrong sentence pairs from newspaper edit logs and artificially generated data. The method involves extract- ing sentences containing edits of spelling cor- rection from edit logs, using commonly con- fused right-and-wrong word pairs to generate artificial right-and-wrong sentence pairs in or- der to expand our training data, and training the NMT model. The evaluation on the United Daily News (UDN) Edit Logs and SIGHAN-7 Shared Task shows that adding artificial error data significantly improves the performance of Chinese spelling check system. 1 Introduction Spelling check is a common yet important task in natural language processing. It plays an important role in a wide range of applications such as word processors, assisted writing systems, and search en- gines. For example, Web search engines such as Google and Bing typically perform spelling check on queries, in order to retrieve documents better meeting the user’s information need. In contrast to Web search engines, while Microsoft Word has a very effective spelling checker for English, there is still considerable room to improve one for Chi- nese. Compared to Western languages (e.g., English and German), relatively little work has been done on Chinese spelling check because there are more chal- lenges such as unclear word boundaries and massive characters in Chinese. Moreover, lack of training data hinders the development of Chinese spelling check. Consider the sentence “他在文學方面有很高 的造旨。”, in which the character “旨” is a typo. For another sentence “他在文學方面有很高的造 藝。”, the character “藝” is also a typo. The correct character of the two typos should be “詣”. Chinese spelling errors often stem from two main reasons: one is similar sound (e.g., *藝 and 詣) and the other is similar shape (e.g., *旨 and 詣), as pointed by Liu et al. (2011). Intuitively, a spelling error can be automati- cally corrected more precisely with machine learn- ing models trained on more data. Unfortunately, there is only limited training data of spelling cor- rection in Chinese, and thus it is no easy to train an NMT model achieving good performance of Chi- nese spelling check. Therefore, we want to improve the spelling check model by generating artificial er- rors to increase training data. In this paper, we present a new system, AccuSpell, that automatically learns to generate the corrected sentence for a potentially misspelled sentence using neural machine translation (NMT) model. An ex- ample of AccuSpell checking for the sentence “在我 們的生命中，常會碰到一些措折和失敗。” is shown in Figure 1. AccuSpell learns how to effec- tively correct a given sentence by training on more data, including real edit …

Content truncated to 500 words