Measuring Contextual Fitness Using Error Contexts Extracted from the Wikipedia Revision History

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 529–538, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Measuring Contextual Fitness Using Error Contexts Extracted from the Wikipedia Revision History Torsten Zesch Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research and Educational Information, Frankfurt Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt http://www.ukp.tu-darmstadt.de Abstract We evaluate measures of contextual fitness on the task of detecting real-word spelling errors. For that purpose, we extract nat- urally occurring errors and their contexts from the Wikipedia revision history. We show that such natural errors are better suited for evaluation than the previously used artificially created errors. In partic- ular, the precision of statistical methods has been largely over-estimated, while the precision of knowledge-based approaches has been under-estimated. Additionally, we show that knowledge-based approaches can be improved by using semantic relatedness measures that make use of knowledge be- yond classical taxonomic relations. Finally, we show that statistical and knowledge- based methods can be combined for in- creased performance. 1 Introduction Measuring the contextual fitness of a term in its context is a key component in different NLP ap- plications like speech recognition (Inkpen and D´esilets, 2005), optical character recognition (Wick et al., 2007), co-reference resolution (Bean and Riloff, 2004), or malapropism detection (Bol- shakov and Gelbukh, 2003). The main idea is al- ways to test what fits better into the current con- text: the actual term or a possible replacement that is phonetically, structurally, or semantically simi- lar. We are going to focus on malapropism detec- tion as it allows evaluating measures of contex- tual fitness in a more direct way than evaluating in a complex application which always entails in- fluence from other components, e.g. the quality of the optical character recognition module (Walker et al., 2010). A malapropism or real-word spelling error oc- curs when a word is replaced with another cor- rectly spelled word which does not suit the con- text, e.g. “People with lots of honey usually live in big houses.”, where ‘money’ was replaced with ‘honey’. Besides typing mistakes, a major source of such errors is the failed attempt of au- tomatic spelling correctors to correct a misspelled word (Hirst and Budanitsky, 2005). A real-word spelling error is hard to detect, as the erroneous word is not misspelled and fits syntactically into the sentence. Thus, measures of contextual fitness are required to detect words that do not fit their contexts. Existing measures of contextual fitness can be categorized into knowledge-based (Hirst and Bu- danitsky, 2005) and statistical methods (Mays et al., 1991; Wilcox-OHearn et al., 2008). Both test the lexical cohesion of a word with its con- text. For that purpose, knowledge-based ap- proaches employ the structural knowledge en- coded in lexical-semantic networks like WordNet (Fellbaum, 1998), while statistical approaches rely on co-occurrence counts collected from large corpora, e.g. the Google Web1T corpus (Brants and Franz, 2006). So far, evaluation of contextual …

Content truncated to 500 words