From N-Grams to Collocations: An Evaluation of Xtract

FROM N-GRAMS TO COLLOCATIONS AN EVALUATION OF XTRACT Frank A. Smadja Department of Computer Science Columbia University New York, NY 10027 Abstract In previous papers we presented methods for retrieving collocations from large samples of texts. We described a tool, Xtract, that im- plements these methods and able to retrieve a wide range of collocations in a two stage process. These methods a.s well as other re- lated methods however have some limitations. Mainly, the produced collocations do not in- clude any kind of functional information and many of them are invalid. In this paper we introduce methods that address these issues. These methods are implemented in an added third stage to Xtract that examines the set of collocations retrieved during the previous two stages to both filter out a number of invalid col- locations and add useful syntactic information to the retained ones. By combining parsing and statistical techniques the addition of this third stage has raised the overall precision level of Xtract from 40% to 80% With a precision of 94%. In the paper we describe the methods and the evaluation experiments. 1 INTRODUCTION In the past, several approaches have been proposed to retrieve various types of collocations from the analysis of large samples of textual data. Pairwise associations (bigrams or 2-grams) (e.g., [Smadja, 1988], [Church and Hanks, 1989]) as well as n-word (n > 2) associations (or n-grams) (e.g., [Choueka el al., 1983], [Smadja and McKeown, 1990]) were retrieved. These techniques auto- matically produced large numbers of collocations along with statistical figures intended to reflect their relevance. However, none of these techniques provides functional in- formation along with the collocation. Also, the results produced often contained improper word associations re- flecting some spurious aspect of the training corpus that did not stand for true collocations. This paper addresses these two problems. Previous papers (e.g., [Smadja and McKeown, 1990]) introduced a. set of tecl)niques and a. tool, Xtract, that produces various types of collocations from a two- stage statistical analysis of large textual corpora briefly sketched in the next section. In Sections 3 and 4, we show how robust parsing technology can be used to both filter out a number of invalid collocations as well as add useful syntactic information to the retained ones. This filter/analyzer is implemented in a third stage of Xtract that automatically goes over a the output collocations to reject the invalid ones and label the valid ones with syn- tactic information. For example, if the first two stages of Xtract produce the collocation "make-decision," the goal of this third stage'is to identify it as a verb-object collocation. If no such syntactic relation is observed, then the collocation is rejected. In Section 5 we present an evaluation of Xtract as a collocation retrieval sys- tem. The addition of the third stage of Xtract has been evaluated to raise the precision of Xtract from 40% to 80°£ and it has a recall of 94%. In this paper we use ex- amples related to the word …

Content truncated to 500 words