Disentangled Phonetic Representation for Chinese Spelling Correction

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 13509–13521 July 9-14, 2023 ©2023 Association for Computational Linguistics Disentangled Phonetic Representation for Chinese Spelling Correction Zihong Liang1, Xiaojun Quan1∗, Qifan Wang2 1School of Computer Science and Engineering, Sun Yat-sen University 2Meta AI liangzh63@mail2.sysu.edu.cn, quanxj3@mail.sysu.edu.cn, wqfcr@fb.com Abstract Chinese Spelling Correction (CSC) aims to de- tect and correct erroneous characters in Chinese texts. Although efforts have been made to in- troduce phonetic information (Hanyu Pinyin) in this task, they typically merge phonetic representations with character representations, which tends to weaken the representation effect of normal texts. In this work, we propose to dis- entangle the two types of features to allow for direct interaction between textual and phonetic information. To learn useful phonetic repre- sentations, we introduce a pinyin-to-character objective to ask the model to predict the correct characters based solely on phonetic informa- tion, where a separation mask is imposed to disable attention from phonetic input to text. To avoid overfitting the phonetics, we further design a self-distillation module to ensure that semantic information plays a major role in the prediction. Extensive experiments on three CSC benchmarks demonstrate the superiority of our method in using phonetic information1. 1 Introduction Chinese Spelling Correction (CSC) is a task to de- tect and correct erroneous characters in Chinese sentences, which plays an indispensable role in many natural language processing (NLP) applica- tions (Martins and Silva, 2004; Gao et al., 2010). Previous research (Liu et al., 2010) shows that the misuse of homophonic characters accounts for roughly 83% of the spelling errors. We present two such cases in Table 1. In the first one, the erroneous characters of “户秃” are difficult to be corrected by only literal text because the input sample is too short and the two characters are entirely unrelated to the semantic meaning of this sample. However, their pronunciation easily helps us associate them with the correct answer “糊涂” which shares the same pronunciation as “户秃”. The second case ∗Corresponding authors 1https://github.com/liangzh63/DORM-CSC Source 可是我忘了，我真户秃(hu tu)。 But I forgot, I am so household bald. Target 可是我忘了，我真糊涂(hu tu)。 But I forgot, I am so silly. BERT 可是我忘了，我真护突。 � PinyinBERT 可是我忘了，我真糊涂。 � REALISE 可是我忘了，我真户涂。 � Our DORM 可是我忘了，我真糊涂。 � Source 可是现在我什么事都不济的(ji de)。 But I can’t do anything right now. Target 可是现在我什么事都不记得(ji de)。 But I don’t remember anything now. BERT 可是现在我什么事都不记得。 � PinyinBERT 可是现在我什么事都不记的。 � REALISE 可是现在我什么事都不记的。 � Our DORM 可是现在我什么事都不记得。 � Table 1: Two examples of Chinese Spelling Correction and the predictions by different models. Misspelled characters are highlighted in red and the corresponding answers are in blue. The phonetic transcription of key characters is bracketed. PinyinBERT is a special BERT model which takes as input only phonetic features with- out characters. REALISE is a state-of-the-art model. exhibits a similar phenomenon but is more compli- cated as the model must distinguish between “记 得” and “记的” further. These two examples illus- trate that misspelled characters could be recognized and corrected with the introduction of phonetic in- formation. …

Content truncated to 500 words