Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution

Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 835–842, Vancouver, October 2005. c⃝2005 Association for Computational Linguistics Using the Web as an Implicit Training Set: Application to Structural Ambiguity Resolution Preslav Nakov and Marti Hearst EECS and SIMS University of California at Berkeley Berkeley, CA 94720 nakov@cs.berkeley.edu, hearst@sims.berkeley.edu Abstract Recent work has shown that very large corpora can act as training data for NLP algorithms even without explicit labels. In this paper we show how the use of sur- face features and paraphrases in queries against search engines can be used to infer labels for structural ambiguity resolution tasks. Using unsupervised algorithms, we achieve 84% precision on PP-attachment and 80% on noun compound coordination. 1 Introduction Resolution of structural ambiguity problems such as noun compound bracketing, prepositional phrase (PP) attachment, and noun phrase coordination re- quires using information about lexical items and their cooccurrences. This in turn leads to the data sparseness problem, since algorithms that rely on making decisions based on individual lexical items must have statistics about every word that may be encountered. Past approaches have dealt with the data sparseness problem by attempting to generalize from semantic classes, either manually built or auto- matically derived. More recently, Banko and Brill (2001) have ad- vocated for the creative use of very large text col- lections as an alternative to sophisticated algorithms and hand-built resources. They demonstrate the idea on a lexical disambiguation problem for which la- beled examples are available “for free”. The prob- lem is to choose which of 2-3 commonly confused words (e.g., {principle, principal}) are appropriate for a given context. The labeled data comes “for free” by assuming that in most edited written text, the words are used correctly, so training can be done directly from the text. Banko and Brill (2001) show that even using a very simple algorithm, the results continue to improve log-linearly with more training data, even out to a billion words. A potential limita- tion of this approach is the question of how applica- ble it is for NLP problems more generally – how can we treat a large corpus as a labeled collection for a wide range of NLP tasks? In a related strand of work, Lapata and Keller (2004) show that computing n-gram statistics over very large corpora yields results that are competi- tive with if not better than the best supervised and knowledge-based approaches on a wide range of NLP tasks. For example, they show that for the problem of noun compound bracketing, the perfor- mance of an n-gram based model computed using search engine statistics was not significantly differ- ent from the best supervised algorithm whose pa- rameters were tuned and which used a taxonomy. They find however that these approaches generally fail to outperform supervised state-of-the-art models that are trained on smaller corpora, and so conclude that web-based n-gram statistics should be the base- line to beat. We feel the potential of these ideas is not …

Content truncated to 500 words