Using the Web to Overcome Data Sparseness

Using the Web to Overcome Data Sparseness Frank Keller and Maria Lapata Division of Informatics University of Edinburgh 2 Buccleuch Place Edinburgh, EH8 9LW, UK {keller, mlap}@cogsci.ed.ac.uk Olga Ourioupina Department of Computational Linguistics Saarland University PO Box 15 11 50 66041 Saarbr¨ucken, Germany ourioupi@coli.uni-sb.de Abstract This paper shows that the web can be em- ployed to obtain frequencies for bigrams that are unseen in a given corpus. We describe a method for retrieving counts for adjective-noun, noun-noun, and verb- object bigrams from the web by querying a search engine. We evaluate this method by demonstrating that web frequencies and correlate with frequencies obtained from a carefully edited, balanced corpus. We also perform a task-based evaluation, showing that web frequencies can reliably predict human plausibility judgments. 1 Introduction In two recent papers, Banko and Brill (2001a; 2001b) criticize the fact that current NLP algo- rithms are typically optimized, tested, and compared on fairly small data sets (corpora with millions of words), even though data sets several orders of mag- nitude larger are available, at least for some tasks. Banko and Brill go on to demonstrate that learning algorithms typically used for NLP tasks benefit sig- nificantly from larger training sets, and their perfor- mance shows no sign of reaching an asymptote as the size of the training set increases. Arguably, the largest data set that is available for NLP is the web, which currently consists of at least 968 million pages.1 Data retrieved from the web therefore provides enormous potential 1This is the number of pages indexed by Google in March 2002, as estimated by Search Engine Showdown (see http://www.searchengineshowdown.com/). for training NLP algorithms, if Banko and Brill’s findings generalize. There is a small body of existing research that tries to harness the potential of the web for NLP. Grefenstette and Nioche (2000) and Jones and Ghani (2000) use the web to generate corpora for languages where elec- tronic resources are scarce, while Resnik (1999) describes a method for mining the web for bilin- gual texts. Mihalcea and Moldovan (1999) and Agirre and Martinez (2000) use the web for word sense disambiguation, and Volk (2001) proposes a method for resolving PP attachment ambiguities based on web data. A particularly interesting application is pro- posed by Grefenstette (1998), who uses the web for example-based machine translation. His task is to translate compounds from French into English, with corpus evidence serving as a filter for candi- date translations. As an example consider the French compound groupe de travail. There are five transla- tion of groupe and three translations for travail (in the dictionary that Grefenstette (1998) is using), re- sulting in 15 possible candidate translations. Only one of them, viz., work group has a high corpus frequency, which makes it likely that this is the correct translation into English. Grefenstette (1998) observes that this approach suffers from an acute data sparseness problem if the corpus counts are obtained from a conventional corpus such as the British National Corpus (BNC) (Burnard, 1995). However, as …

Content truncated to 500 words