A Categorial Variation Database for English

A Categorial Variation Database for English Nizar Habash Institute for Advanced Computer Studies University of Maryland College Park, MD 20740 habash@umiacs.umd.edu Bonnie Dorr Institute for Advanced Computer Studies University of Maryland College Park, MD 20740 bonnie@umiacs.umd.edu Abstract We describe our approach to the construction and evaluation of a large-scale database called “CatVar” which contains categorial variations of English lexemes. Due to the prevalence of cross-language categorial variation in multilin- gual applications, our categorial-variation re- source may serve as an integral part of a di- verse range of natural language applications. Thus, the research reported herein overlaps heavily with that of the machine-translation, lexicon-construction, and information-retrieval communities. We apply the information-retrieval metrics of precision and recall to evaluate the accuracy and coverage of our database with respect to a human-produced gold standard. This evaluation reveals that the categorial database achieves a high degree of precision and recall. Additionally, we demonstrate that the database improves on the linkability of Porter stemmer by over 30%. 1 Introduction Natural Language Processing (NLP) applications may only be as good as the resources upon which they rely. Resources specifying the relations among lexical items such as WordNet (Fellbaum, 1998) and HowNet (Dong, 2000) (among others) have inspired the work of many re- searchers in NLP (Carpuat et al., 2002; Dorr et al., 2000; Resnik, 1999; Hearst, 1998; Voorhees, 1993). In this paper we introduce a new resource called Cat- Var which specifies the lexical relation Categorial Vari- ation on a large scale for English. This resource has al- ready been used effectively in a wide range of monolin- gual and multilingual NLP applications. Upon its first public release, CatVar will be freely available to the re- search community. We expect that the contribution of this resource will become more widely recognized through its future incorporation into additional NLP applications. A categorial variation of a word with a certain part- of-speech is a derivationally-related word with possi- bly a different part-of-speech. For example, hunger � , hunger � and hungry �� are categorial variations of each other, as are cross � and across � , and stab � and stab � . Although this relation seems basic on the surface, this relation is critical to work in Information Retrieval (IR), Natural Language Generation (NLG) and Machine Trans- lation (MT)—yet there is no large scale resource avail- able for English that focuses on categorial variations.1 In the rest of this paper, we discuss other available re- sources and how they differ from the CatVar database. We then discuss how and what resources were used to build CatVar. Afterwards, we present three applications that use CatVar in different ways: Generation-Heavy MT, headline generation, and cross-language divergence un- raveling for bilingual alignment. Finally, we present a multi-component evaluation of the database. Our evalu- ation reveals that the categorial database achieves a high degree of precision and recall and that it improves on the linkability of Porter stemmer by over 30%. 2 Background Lexical relations describe relative relationships among different lexemes. …

Content truncated to 500 words