Correcting the Autocorrect: Context-Aware Typographical Error Correction via Training Data Augmentation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6930–6936 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6930 Correcting the Autocorrect: Context-Aware Typographical Error Correction via Training Data Augmentation Kshitij Shah, Gerard de Melo Department of Computer Science Rutgers University–New Brunswick kshitij.shah@rutgers.edu, gdm@demelo.org Abstract In this paper, we explore the artificial generation of typographical errors based on real-world statistics. We first draw on a small set of annotated data to compute spelling error statistics. These are then invoked to introduce errors into substantially larger corpora. The generation methodology allows us to generate particularly challenging errors that require context-aware error detection. We use it to create a set of English language error detection and correction datasets. Finally, we examine the effectiveness of machine learning models for detecting and correcting errors based on this data. Keywords: Corpus, Error Generation, Deep Learning 1. Introduction With the emergence of the Internet as the preferred medium for communication and the rise of social media, the number of words typed by an average human being has increased manyfold in the last decades. The majority of this com- munication is casual in nature, with less attention and time dedicated by the writer compared to formal writing. This leads to a plethora of typographical errors in such text. This problem is exacerbated by the comparably small size of the keyboard on hand-held devices. Hence, typographical er- rors, which were somewhat more limited and likely to be corrected immediately in past decades, are now ubiquitous. Traditional solutions such as dictionary-based spelling cor- rection are often inadequate. First of all, online writing tends to make liberal use of neologisms, slang words, and so on, that are lacking in pre-existing dictionaries. Any au- tomated means to mine dictionaries from social media risk also incorporating frequent misspellings such as *seperate into the dictionary. Second, a higher error rate also entails a larger likelihood of a typographical error leading to a form that happens to be a legitimate word in the dictionary. This problem exists even with more advanced spelling correction software, as indeed, many such errors are now caused by autocorrection software. To overcome these challenges, an alternative is to draw on advances in machine learning to identify and correct such ty- pographical errors. While autocorrection typically considers just the current and possibly the last few words, deep models that can account for both sides of a larger context window have the potential to more accurately determine whether a word fits in context. Unfortunately, deep learning generally requires large annotated corpora for effective training. Such large annotated datasets are difficult to procure, as the la- beling process tends to be time-consuming and expensive (Qiu et al., 2020). There have been significant efforts in the last decade to overcome this gap for the task of grammatical error correction (Dale and Kilgarriff, 2011; Ng et al., 2013; Ng et al., 2014). However, no sufficiently large datasets exist for typographical error correction. In this paper, …

Content truncated to 500 words