Using Unknown Word Techniques to Learn Known Words

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 902–912, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Using Unknown Word Techniques To Learn Known Words Kostadin Cholakov University of Groningen The Netherlands k.cholakov@rug.nl Gertjan van Noord University of Groningen The Netherlands g.j.m.van.noord@rug.nl Abstract Unknown words are a hindrance to the perfor- mance of hand-crafted computational gram- mars of natural language. However, words with incomplete and incorrect lexical entries pose an even bigger problem because they can be the cause of a parsing failure despite being listed in the lexicon of the grammar. Such lex- ical entries are hard to detect and even harder to correct. We employ an error miner to pinpoint words with problematic lexical entries. An auto- mated lexical acquisition technique is then used to learn new entries for those words which allows the grammar to parse previously uncovered sentences successfully. We test our method on a large-scale grammar of Dutch and a set of sentences for which this grammar fails to produce a parse. The appli- cation of the method enables the grammar to cover 83.76% of those sentences with an ac- curacy of 86.15%. 1 Introduction In this paper, we present an automated two-phase method for treating incomplete or incorrect lexical entries in the lexicons of large-scale computational grammars. The performance of our approach is tested in a case study with the wide-coverage Alpino grammar (van Noord, 2006) of Dutch. When ap- plied to real test sentences previously not covered by Alpino, the method causes a parsing coverage of 83.76% and the accuracy of the delivered analyses is 86.15%. The main advantage of our approach is the suc- cessful combination of efficient error mining and lexical acquisition techniques. In the first phase, er- ror mining pinpoints words which are listed in the lexicon of a given grammar but which nevertheless often lead to a parsing failure. This indicates that the current lexical entry for such a word is either wrong or incomplete and that one or more correct entries for this word are missing from the lexicon. Our idea is to treat the word as if it was unknown and, in the second phase, to employ lexical acquisition (LA) to learn the missing correct entries. In the case study presented here, we employ the iterative error miner of de Kok et al. (2009). Since it has to be run on a large parsed corpus, we have parsed the Flemish Mediargus corpus (∼1.5 billion words) with Alpino. The reason for this choice is the relatively large lexical difference between stan- dard Dutch and Flemish. This increases the chance to encounter words which are used in Flemish in a way not handled by Alpino yet. For example, the word afwater (to drain) is listed as a first person singular present verb in the Alpino lexicon. However, the error miner identifies this word as the reason for the parsing failure of 9 sen- tences. A manual examination reveals that the word …

Content truncated to 500 words