Confusionset-guided Pointer Networks for Chinese Spelling Check

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5780–5785 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5780 Confusionset-guided Pointer Networks for Chinese Spelling Check Dingmin Wang♣, Yi Tay♠, Li Zhong♣ ♣Tencent Cloud AI ♠ Nanyang Technological University, Singapore wangdimmy@gmail.com ytay017@e.ntu.edu.sg reggiezhong@tencent.com Abstract This paper proposes Confusionset-guided Pointer Networks for Chinese Spell Check (CSC) task. More concretely, our approach utilizes the off-the-shelf confusionset for guid- ing the character generation. To this end, our novel Seq2Seq model jointly learns to copy a correct character from an input sen- tence through a pointer network, or generate a character from the confusionset rather than the entire vocabulary. We conduct experiments on three human-annotated datasets, and re- sults demonstrate that our proposed generative model outperforms all competitor models by a large margin of up to 20% F1 score, achieving state-of-the-art performance on three datasets. 1 Introduction In our everyday writing, there exists different types of errors, one of which that frequently oc- curs is misspelling a character due to the charac- ters’ similarity in terms of sound, shape, and/or meaning. Spelling check is a task to detect and correct such problematic usage of language. Al- though these tools been useful, detecting and fix- ing errors in natural language, especially in Chi- nese, remains far from solved. Notably, Chinese is very different from other alphabetical languages (e.g., English). First, there are no word delimiters between the Chinese words. Second, the error de- tection task is difficult due to its context-sensitive nature, i.e., errors can be only often determined at phrase/sentence level and not at character-level. In this paper, we propose a novel neural archi- tecture for the Chinese Spelling Check (CSC) task. For the task at hand, it is intuitive that the gener- ated sentence and the input sentence would usu- ally share most characters, along with same sen- tence structure with a slight exception for several incorrect characters. This is unlike other genera- tive tasks (e.g., neural machine translation or di- alog translation) in which the output would differ greatly from the input. To this end, this paper proposes a novel Confusionset-guided copy mechanism which achieves significant performance gain over com- petitor approaches. Copy mechanisms (Gulcehre et al., 2016), enable the copying of words di- rectly from the input via pointing, providing an extremely appropriate inductive bias for the CSC task. More concretely, our model jointly learns the selection of appropriate characters to copy or to generate a correct character from the vocabulary when an incorrect character occurs. The clear novelty of our work, however, is the infusion of Confusionsets1 with Pointer Networks, which help reduce the search space and vastly improve the probability of generating correct characters. Experimental results on three benchmark datasets demonstrate that our model outperforms all competitor models, obtaining performance gains of up to 20%. 2 Our Proposed Model Given an input, we represent the input sentence as X = {cs 1, cs 2, · · · , cs n}, …

Content truncated to 500 words