Experiments on Candidate Data for Collocation Extraction

Experiments on Candidate Data for Collocation Extraction Stefan Evert and Hannah Kermes Institut fiir Maschinelle Sprachverarbeitung Universitat Stuttgart {evert,kermes}@ims.uni - stuttgart.de Abstract The paper describes ongoing work on the evaluation of methods for extract- ing collocation candidates from large text corpora. Our research is based on a German treebank corpus used as gold standard. Results are available for adjective+noun pairs, which proved to be a comparatively easy extraction task. We plan to extend the evalua- tion to other types of collocations (e.g., PP+verb pairs). 1 Introduction While a mostly British tradition based on the ideas of J. R. Firth defines collocations as (sig- nificantly) frequent combinations of words cooc- curring within a given text span, applications in terminology, lexicography, and natural language processing prefer a more restricted view. Collo- cations are understood as unpredictable combina- tions of words in a particular (morpho-)syntactic relation (adjectives modifying nouns, direct ob- jects of verbs, or English noun-noun compounds). The extraction of such collocations from text cor- pora is usually performed in a three-stage process (cf. Krenn (2000, 28-32) and references therein): 1. The source corpus is annotated with vary- ing amounts of linguistic information (rang- ing from part-of-speech tags to full parse trees), depending on the tools available. Then a list of word pairs satisfying the required (morpho-)syntactic constraints is extracted (typically based on part-of-speech patterns). This first candidate list will contain both col- locational and non-collocational pairs. 2. Linguistic and/or heuristic filters may be ap- plied to reduce the size of the candidate set. For instance, certain "generic" adjectives as well as those derived from verb participles are rarely found in adj+noun collocations. 3. The remaining candidates are ranked by sta- tistical measures based on their frequency "profiles". Usually, word pairs are consid- ered likely to be collocations if their cooccur- rence frequency is much higher than expected by chance. Authors typically evaluate the performance of a single collocation extraction system as a whole (e.g. Smadj a (1993)). A small number of in-depth comparative evaluations (mostly Daille (1994) and Krenn (2000)) concentrate on the quality of the statistical measures and the corresponding ranking of the candidates, and to a lesser extent on the performance of linguistic filters. Although Evert and Krenn (2001) are aware of the influence that the first extraction step has on their results, they fail to give a quantitative evaluation of differ- ent pre-processing and extraction methods. Our research aims to fill this gap. Currently, we are evaluating methods for the extraction of adjec- tive+noun pairs from German newspaper text. It is planned to extend our work to other types of collo- cations, including PP+verb and noun+verb pairs. 83 2 Evaluation procedure With a collocation definition that is not based purely on observed frequencies, the statistical ranking of candidates has to be evaluated against a manually confirmed list of true positives (cf. Daille (1994) and Krenn (2000)). This methodol- ogy is of little use for the evaluation of the candi- date extraction step, though, for several reasons: • The …

Content truncated to 500 words