Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 716–728 August 1–6, 2021. ©2021 Association for Computational Linguistics 716 Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking Heng-Da Xu1∗, Zhongli Li2∗, Qingyu Zhou2, Chao Li2, Zizhen Wang2, Yunbo Cao2, Heyan Huang1, Xian-Ling Mao1 1Beijing Institute of Technology 2Tencent Cloud Xiaowei {xuhengda,hhy63,maoxl}@bit.edu.cn {neutrali,qingyuzhou,diegoli,zizhenwang,yunbocao}@tencent.com Abstract Chinese Spell Checking (CSC) aims to de- tect and correct erroneous characters for user- generated text in Chinese language. Most of the Chinese spelling errors are misused se- mantically, phonetically or graphically similar characters. Previous attempts notice this phe- nomenon and try to utilize the similarity rela- tionship for this task. However, these meth- ods use either heuristics or handcrafted confu- sion sets to predict the correct character. In this paper, we propose a Chinese spell checker called REALISE, by directly leveraging the multimodal information of the Chinese char- acters. The REALISE model tackles the CSC task by (1) capturing the semantic, phonetic and graphic information of the input charac- ters, and (2) selectively mixing the informa- tion in these modalities to predict the correct output. Experiments1 on the SIGHAN bench- marks show that the proposed model outper- forms strong baselines by a large margin. 1 Introduction The Chinese Spell Checking (CSC) task aims to identify erroneous characters and generate candi- dates for correction. It has attracted much research attention, due to its fundamental and wide appli- cations such as search query correction (Martins and Silva, 2004; Gao et al., 2010), optical character recognition (OCR) (Afli et al., 2016), automatic essay scoring (Dong and Zhang, 2016). Recently, rapid progress (Zhang et al., 2020; Cheng et al., 2020) has been made on this task, because of the success of large pretrained language models (De- vlin et al., 2019; Liu et al., 2019; Yang et al., 2019). In alphabetic languages such as English, spelling errors often occur owing to one or more wrong ∗Heng-Da Xu and Zhongli Li contributed equally. Work is done during internship at Tencent Cloud Xiaowei. Qingyu Zhou is the corresponding author. 1Code and model are available at https://github. com/DaDaMrX/ReaLiSe. Phonetically Similar Case Sent. 晚饭后他递给我一平(p´ıng, flat)红酒。 晚饭后他递给我一杯(b¯ei, cup)红酒。  Cand. 晚饭后他递给我一瓶(p´ıng, bottle)红酒。  晚饭后他递给我一箱(xi¯ang, box)红酒。  Trans. He handed me a bottle of red wine after dinner. Graphically Similar Case Sent. 每天放学我都会轻(q¯ıng, light)过这片树林。 每天放学我都会路(l`u, pass)过这片树林。  Cand. 每天放学我都会经(j¯ıng, go)过这片树林。  每天放学我都会走(zˇou, walk)过这片树林。  Trans. I go through this wood every day after school. Table 1: Two examples of Chinese spelling errors and their candidate corrections. “Sent./Cand./Trans.” are short for sentence/candidates/translation respectively. The wrong/candidate/correct characters with their pro- nunciation and translation are in red/orange/blue color. characters, resulting in the written word not in the dictionary problem (Tachibana and Komachi, 2016). However, Chinese characters are valid if they can be typed in computer systems, which causes that the spelling errors are de facto mis- used characters in the context of computer-based language processing. Considering the formation of Chinese characters, a few of them were originally pictograms or phono-semantic compound charac- ters (Jerry, 1988). Thus, …

Content truncated to 500 words