Lexical Patterns or Dependency Patterns: Which Is Better for Hypernym Extraction?

Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pages 174–182, Boulder, Colorado, June 2009. c⃝2009 Association for Computational Linguistics Lexical Patterns or Dependency Patterns: Which Is Better for Hypernym Extraction? Erik Tjong Kim Sang Alfa-informatica University of Groningen e.f.tjong.kim.sang@rug.nl Katja Hofmann ISLA, Informatics Institute University of Amsterdam khofmann@science.uva.nl Abstract We compare two different types of extraction patterns for automatically deriving semantic information from text: lexical patterns, built from words and word class information, and dependency patterns with syntactic informa- tion obtained from a full parser. We are partic- ularly interested in whether the richer linguis- tic information provided by a parser allows for a better performance of subsequent informa- tion extraction work. We evaluate automatic extraction of hypernym information from text and conclude that the application of depen- dency patterns does not lead to substantially higher precision and recall scores than using lexical patterns. 1 Introduction For almost a decade, automatic sentence parsing systems with a reasonable performance (90+% con- stituent precision/recall) have been available for En- glish (Charniak, 1999). In recent years there has been an increase in linguistic applications which use parsing as a preprocessing step, e.g. Snow et al. (2006) and Surdeanu et al. (2008). One of the boosts for these new applications was the increasing power of desktop computers, which allows for an easier ac- cess to the computing-intensive parsing results. An- other is the increased popularity of dependency pars- ing of which the results can easily be incorporated into followup systems. Although there is a consensus about the fact that the richness of the dependency structures should, in principle, enable better performance than lexical in- formation or shallow parsing results, it is not clear if these better results can also be obtained in practice. A performance of 90% precision and recall at con- stituent level still leaves an average of one error in a medium-length sentence of ten words. These er- rors could degrade the performance of any approach which relies heavily on parser output. The question of whether to include a full parser as a preprocessor for natural language processing task, has led to a heated discussion between the two au- thors of the paper. One of us argues that full parsers are slow and make too many errors, and relies on shallow techniques like part-of-speech tagging for preprocessing. The other points at the decreasing costs of computing and improvements in the reliabil- ity of parsers, and recommends dependency parsers as preprocessing tools. While no automatic text preprocessing method is free of errors, it is indeed true that approaches other than full parsing, like for example shallow parsing, offer useful information at a considerably cheaper processing cost. The choice between using a heavy full parser or a light shallow language analyzer is one that developers of language processing systems frequently have to make. The expected performance boost of parsed data could be an important motiva- tion for choosing for full syntactic analysis. How- ever, we do not know how big the …

Content truncated to 500 words