Exploiting Syntactic and Distributional Information for Spelling Correction with Web-Scale N-gram Models

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1291–1300, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Exploiting Syntactic and Distributional Information for Spelling Correction with Web-Scale N-gram Models Wei Xuc,∗Joel Tetreaulta Martin Chodorowb Ralph Grishmanc Le Zhaod aEducational Testing Service, Princeton, NJ, USA jtetreault@ets.org bHunter College of CUNY, New York, NY, USA martin.chodorow@hunter.cuny.edu cNew York University, NY, USA {xuwei,grishman}@cs.nyu.edu dCarnegie Mellon University, Pittsburgh, PA, USA lezhao@cs.cmu.edu Abstract We propose a novel way of incorporating de- pendency parse and word co-occurrence in- formation into a state-of-the-art web-scale n- gram model for spelling correction. The syn- tactic and distributional information provides extra evidence in addition to that provided by a web-scale n-gram corpus and especially helps with data sparsity problems. Experimental results show that introducing syntactic fea- tures into n-gram based models significantly reduces errors by up to 12.4% over the current state-of-the-art. The word co-occurrence in- formation shows potential but only improves overall accuracy slightly. 1 Introduction The function of context-sensitive text correction is to identify word-choice errors in text (Bergsma et al., 2009). It can be viewed as a lexical disambigua- tion task (Lapata and Keller, 2005), where a system selects from a predefined confusion word set, such as {affect, effect} or {complement, compliment}, and provides the most appropriate word choice given the context. Typically, one determines if a word has been used correctly based on lexical, syntactic and semantic information from the context of the word. One of the top performing models of spelling cor- rection (Bergsma et al., 2010) is based on web-scale n-gram counts, which reflect both syntax and mean- ing. However, even with a large-scale n-gram cor- pus, data sparsity can hurt performance in two ways. ∗This work was done when the first author was an intern for Educational Testing Service. First, n-gram based methods require exact word and order matches. If there is a low frequency word in the context, such as a person’s name, there will be little, if any, evidence in the n-gram data to sup- port the usage. Second, if the target confusable word is rare, there will not be enough n-gram support or training data to render a confident decision. Because of the data sparsity problem, language modeling is not always sufficient to capture the meaning of the sentence and the correct usage of the word. Take a sentence from The New York Times (NYT) for example: “‘This fellow’s won a war,’ the dean of the capital’s press corps, David Broder, an- nounced on ‘Meet the Press’ after complimenting the president on the ‘great sense of authority and command’ he exhibited in a flight suit.” Unfortu- nately, neither the phrase “complementing the pres- ident” nor “complimenting the president” exists in the web-scale Google N-gram corpus (Brants and Franz, 2006). The n-gram models decide solely based on the frequency of the bi-grams “after com- ple(i)menting” and “comple(i)menting the”, which are common usages for both words. The real ques- tion is whether we are …

Content truncated to 500 words