Portable Spelling Corrector for a Less-Resourced Language: Amharic

Portable Spelling Corrector for a Less-Resourced Language: Amharic Andargachew Mekonnen Gezmu, Andreas Nürnberger, Binyam Ephrem Seyoum Otto-von-Guericke-Universität Magdeburg, Addis Ababa University Otto-von-Guericke-Universität Magdeburg, Fakultät für Informatik, Data and Knowledge Engineering Group, Universitätsplatz 2, 39106 Magdeburg. {andargachew.gezmu, andreas.nuernberger}@ovgu.de, binyam.ephrem@aau.edu.et Abstract This paper describes an automatic spelling corrector for Amharic, the working language of the Federal Government of Ethiopia. We used a corpus-driven approach with the noisy channel for spelling correction. It infers linguistic knowledge from a text corpus. The approach can be ported to other written languages with little effort as long as they are typed using a QWERTY keyboard with direct mappings between keystrokes and characters. Since Amharic letters are syllabic, we used a modified version of the System for Ethiopic Representation in ASCII for transliteration in the like manner as most Amharic keyboard input methods do. The proposed approach is evaluated with Amharic and English test data and has scored better performance result than the baseline systems: GNU Aspell and Hunspell. We get better result due to the smoothed language model, the generalized error model and the ability to take into account the context of misspellings. Besides, instead of using a handcrafted lexicon for spelling error detection, we used a term list derived from frequently occurring terms in a text corpus. Such a term list, in addition to ease of compilation, has also an advantage in handling rare terms, proper nouns, and neologisms. Keywords: spelling corrector, corpora, noisy channel, less-resourced language 1. Introduction Documents in many languages have been digitized and are available in different media especially on the web. Giant software vendors (e.g., Google and Microsoft) are also localizing their products to the native languages of their target customers. There is a need to develop computational solutions to the classic problems of computational linguistics for the respective languages. Spelling error detection and correction are among the oldest computational linguistics problems (Blair, 1960). Spelling correction is considered from two perspectives: non-word and real-word correction. When typographical or cognitive errors accidentally produce valid words we get real-word errors, otherwise, we get non-word errors. These problems are mostly treated separately. In this paper, we dealt with non-word errors. Many spelling correctors are developed using rule-based approaches. However, it is difficult to develop and maintain all language-dependent rules (Norvig, 2009). In addition, such systems limit themselves to isolated-word correction without considering the context. Therefore, we proposed and evaluated an approach that takes into account the context of misspellings and infers linguistics knowledge from a text corpus. 2. Related Work Earliest research on spelling correction is based on phonetic and string similarities such as Metaphone and Damerau-Levenshtein edit distance algorithms (Damerau, 1964). Candidate corrections are ranked from manually compiled lexicons with the help of these algorithms. GNU Aspell and Hunspell are good examples that follow this approach. Mekonnen (2012) has followed the same approach for Amharic. In a related approach, Ahmed et al. (2009) used similarity scores of letter n-grams to rank candidate corrections. In these approaches, the lexicons along with some linguistics …

Content truncated to 500 words