Pronunciation Modeling for Improved Spelling Correction

Pronunciation Modeling for Improved Spelling Correction Kristina Toutanova Computer Science Department Stanford University Stanford, CA 94305 USA Robert C. Moore Microsoft Research One Microsoft Way Redmond, WA 98052 USA Abstract This paper presents a method for incor- porating word pronunciation information in a noisy channel model for spelling cor- rection. The proposed method builds an explicit error model for word pronuncia- tions. By modeling pronunciation simi- larities between words we achieve a sub- stantial performance improvement over the previous best performing models for spelling correction. 1 Introduction Spelling errors are generally grouped into two classes (Kuckich, 1992) — typographic and cogni- tive. Cognitive errors occur when the writer does not know how to spell a word. In these cases the misspelling often has the same pronunciation as the correct word ( for example writing latex as latecks). Typographic errors are mostly errors related to the keyboard; e.g., substitution or transposition of two letters because their keys are close on the keyboard. Damerau (1964) found that 80% of misspelled words that are non-word errors are the result of a sin- gle insertion, deletion, substitution or transposition of letters. Many of the early algorithms for spelling correction are based on the assumption that the cor- rect word differs from the misspelling by exactly one of these operations (M. D. Kernigan and Gale, 1990; Church and Gale, 1991; Mayes and F. Dam- erau, 1991). By estimating probabilities or weights for the different edit operations and conditioning on the left and right context for insertions and deletions and allowing multiple edit operations, high spelling correction accuracy has been achieved. At ACL 2000, Brill and Moore (2000) introduced a new error model, allowing generic string-to-string edits. This model reduced the error rate of the best previous model by nearly 50%. It proved advantageous to model substitutions of up to 5-letter sequences (e.g ent being mistyped as ant, ph as f, al as le, etc.) This model deals with phonetic errors significantly better than previous models since it allows a much larger context size. However this model makes residual errors, many of which have to do with word pronunciation. For example, the following are triples of misspelling, correct word and (incorrect) guess that the Brill and Moore model made: edelvise edelweiss advise bouncie bouncy bounce latecks latex lacks In this work we take the approach of modeling phonetic errors explicitly by building a separate er- ror model for phonetic errors. More specifically, we build two different error models using the Brill and Moore learning algorithm. One of them is a letter-based model which is exactly the Brill and Moore model trained on a similar dataset. The other is a phone-sequence-to-phone-sequence error model trained on the same data as the first model, but using the pronunciations of the correct words and the es- timated pronunciations of the misspellings to learn phone-sequence-to-phone-sequence edits and esti- mate their probabilities. At classification time, N- best list predictions of the two models are combined using a log linear model. A requirement …

Content truncated to 500 words