An In-Depth Comparison of 14 Spelling Correction Tools on a Common Benchmark

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1849–1857 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1849 An In-Depth Comparison of 14 Spelling Correction Tools on a Common Benchmark Markus Näther University of Freiburg 79110 Freiburg, Germany naetherm@cs.uni-freiburg.de Abstract Determining and correcting spelling and grammar errors in text is an important but surprisingly difficult task. There are several reasons why this remains challenging. Errors may consist of simple typing errors like deleted, substituted, or wrongly inserted letters, but may also consist of word confusions where a word was replaced by another one. In addition, words may be erroneously split into two parts or get concatenated. Some words can contain hyphens, because they were split at the end of a line or are compound words with a mandatory hyphen. In this paper, we provide an extensive evaluation of 14 spelling correction tools on a common benchmark. In particular, the evaluation provides a detailed comparison with respect to 12 error categories. The benchmark consists of sentences from the English Wikipedia, which were distorted using a realistic error model. Measuring the quality of an algorithm with respect to these error categories requires an alignment of the original text, the distorted text and the corrected text provided by the tool. We make our benchmark generation and evaluation tools publicly available. Keywords: Benchmark, Evaluation, Spelling Error Correction, SEC, GEC 1. Introduction Text continues to be one of the most popular formats for publishing information and knowledge, either in electronic or non-electronic form. Huge amounts of text are created every day and the variety of writing styles, formatting, struc- tures, and publishing types are almost unlimited. Due to the fact that language is normally written by humans, it might be the case that different kinds of spelling and grammar errors occur within those written texts. For example, consider this sentence: As an artist, Francis Bacon was a late starter and this slightly distorted version: As an artist, Framcis Bcon was a late starter. The distorted version includes typical er- rors like hitting the key m instead of n and omitting a letter. While the human brain is quite good at fixing such errors during reading, the task is surprisingly difficult for comput- ers. If one knows Francis Bacon, one would clearly identify the tokens Framcis Bcon as him. If one does not know him, one would probably detect that the token Framcis is not cor- rect, since Francis is more familiar. The latter token Bcon could also stand for a wearable1 and would hence be con- sidered correct by many spelling correction tools although it does not make much sense in the given sentence. Spelling correction is a fundamental problem in the field of natural language processing. Most algorithms and tools in the field simply assume that the given text is free from errors such as the above and they fail if this is not the case. In par- ticular, this is true for basic …

Content truncated to 500 words