Empirical Term Weighting and Expansion Frequency

Empirical Term Weighting and Expansion Frequency Kyoji Umemura Toyohashi University of Technology Toyohashi Aichi 441-8580 Japan umemura@tut ics. rut. ac. jp Kenneth W. Church AT&T Labs-Research 180 Park Ave., Florham Park, NJ. kwc~research, art. com Abstract We propose an empirical method for estimating term weights directly from relevance judgements, avoiding various standard but potentially trouble- some assumptions. It is common to assume, for ex- ample, that weights vary with term frequency (t f) and inverse document frequency (idf) in a particu- lar way, e.g., tf. idf, but the fact that there are so many variants of this formula in the literature sug- gests that there remains considerable uncertainty about these assumptions. Our method is similar to the Berkeley regression method where labeled rel- evance judgements are fit as a linear combination of (transforms of) t f, idf, etc. Training meth- ods not only improve performance, but also ex- tend naturally to include additional factors such as burstiness and query expansion. The proposed histogram-based training method provides a sim- ple way to model complicated interactions among factors such as t f, idf, burstiness and expansion frequency (a generalization of query expansion). The correct handling of expanded term is realized based on statistical information. Expansion fre- quency dramatically improves performance from a level comparable to BKJJBIDS, Berkeley's en- try in the Japanese NACSIS NTCIR-1 evaluation for short queries, to the level of JCB1, the top system in the evaluation. JCB1 uses sophisti- cated (and proprietary) natural language process- ing techniques developed by Just System, a leader in the Japanese word-processing industry. We are encouraged that the proposed method, which is simple to understand and replicate, can reach this level of performance. 1 Introduction An empirical method for estimating term weights directly from relevance judgements is proposed. The method is designed to make as few assump- tions as possible. It is similar to Berkeley's use of regression (Cooper et al., 1994) (Chen et al., 1999) where labeled relevance judgements are fit as a linear combination of (transforms of) t f, idf, etc., but avoids potentially troublesome assump- tions by introducing histogram methods. Terms are grouped into bins. Weights are computed based on the number of relevant and irrelevant documents associated with each bin. The result- • t: a term • d: a document • tf(t, d): term freq = # of instances of t in d • df(t): doc freq = # of docs d with tf(t, d) > 1 • N: # of documents in collection • idf(t): inverse document freq: -log2 d~t) • df(t, tel, t f0): # of relevant documents d with tf(t, d) = tfo • df(t, rel, tfo): # of irrelevant documents d with tf(t, d) = tfo • el(t): expansion frequency = # docs d in query expansion with tf(t, d) > 1 • TF(t): standard notion of frequency in corpus-based NLP: TF(t) = ~d tf(t, d) • B(t): burstiness: B(t) = 1 iff ~ is large. df(t) Table 1: Notation ing weights usually lie between 0 …

Content truncated to 500 words