SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 871–881 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 871 SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check Xingyi Cheng∗ Weidi Xu∗ Kunlong Chen Shaohua Jiang Feng Wang Taifeng Wang Wei Chu Yuan Qi Ant Financial Services Group {fanyin.cxy,weidi.xwd,kunlong.ckl,shaohua.jsh,zifan.wf, taifeng.wang,weichu.cw,yuan.qi}@alibaba-inc.com Abstract Chinese Spelling Check (CSC) is a task to de- tect and correct spelling errors in Chinese nat- ural language. Existing methods have made attempts to incorporate the similarity knowl- edge between Chinese characters. However, they take the similarity knowledge as either an external input resource or just heuristic rules. This paper proposes to incorporate phonological and visual similarity knowledge into language models for CSC via a special- ized graph convolutional network (SpellGCN). The model builds a graph over the characters, and SpellGCN is learned to map this graph into a set of inter-dependent character clas- sifiers. These classifiers are applied to the representations extracted by another network, such as BERT, enabling the whole network to be end-to-end trainable. Experiments 1 are conducted on three human-annotated datasets. Our method achieves superior performance against previous models by a large margin. 1 Introduction Spelling errors are common in our daily life, caused typically by human writing, automatic speech recognition, and optical character recognition sys- tems. Among these errors, misspelling a character frequently occurs due to the similarity between characters. In Chinese, many characters are phono- logically and visually similar, but semantically very different. According to Liu et al. (2010), about 83% of errors are related to phonological similarity and 48% are related to visual similarity. The Chi- nese Spelling Check (CSC) task aims to detect and correct such misuse of the Chinese language. De- spite recent development, CSC remains a challeng- ing task. Notably, the spelling checking on Chinese is very different from English, due to its language ∗Equal contribution. 1The dataset and all code for this paper is available at https://github.com/ACL2020SpellGCN/SpellGCN Input 餐厅的换经费产适合约会 (phonics) c¯an t¯ing d¯e hu`an j¯ing f`ei ch´an sh`ı h´e yu¯e hu`ı BERT 餐厅的月消费最适合约会 (phonics) c¯an t¯ing d¯e yu`e xi¯ao f`ei zu`ı sh`ı h´e yu¯e hu`ı Gold Label 餐厅的环境非常适合约会 (phonics) c¯an t¯ing d¯e hu´an j`ıng f¯ei ch´ang sh`ı h´e yu¯e hu`ı Table 1: A CSC data sample from SIGHAN 2014 (Yu et al., 2014) with ID B1-3440-2, the incorrect/correct characters are in orange/blue. A BERT model modifies the text into a sentence that is semantically reasonable but dissimilar in pronunciation. By incorporating both phonological and visual similarities, our new method SpellGCN can generate a sentence that is both seman- tically sensible and phonically similar to the original sentence. The sentence output from SpellGCN means “this restaurant is very suitable for dating”. nature. Chinese is a language consisting of many pictographic characters without word delimiters. And the meaning of each character changes dramat- ically when the context changes. Therefore, a CSC system needs to recognize the semantics and ag- gregate the surrounding information for necessary modifications. Previous methods followed …

Content truncated to 500 words