Weight Estimation for N-Best Rescoring

Weight Estimation for N-Best Rescoring* Ashvin Kannant, Mari Ostendorj]', J. Robin Rohlicekl: t Boston University 44 Cummington St. Boston, MA 02215 :~ BBN Inc. 10 Moulton St. Cambridge, MA 02138 ABSTRACT This paper describes recent improvements in the weight esti- mation technique for sentence hypothesis rescoring using the N-Best formalism. Mismatches between training and test data are also explored. 1. INTRODUCTION The N-Best rescoring paradigm involves the generation of a list of the N best sentence hypotheses by a recog- nition system and the subsequent rescoring of these hy- potheses by other knowledge sources. The sentence hy- potheses are then reranked according to a weighted linear combination of the different scores. This paradigm has the potential of achieving better performance than that of any individual knowledge source, if these scores are combined in an "optimal" manner. This paper discusses the key issues related to estimation of robust weights for a linear combination of scores. 2. WEIGHT ESTIMATION In the initial work [1], the weights used in the linear score combination were chosen to minimize the general- ized mean of the rank of the correct hypothesis using an iterative search algorithm based on Powell's method [2]. Further experience using this technique suggested that the result was very sensitive to the large number of local minima in the optimization criterion. Several steps have been taken to address this issue. The optimization criterion now minimizes the average word error in the top ranking hypothesis. The use of this cri- terion results in a "smoother" weight space, i.e., having fewer local minima. Also addressing the problem of lo- cal minima, we examine a large number of points in the weight space on a lattice spanning the range of proba- ble weights. Powell's method may be used with points on the grid as the initial estimate of weights to find the best performance, or the points on a fine grid may be evaluated directly. The error function is piece-wise constant over the weight *This research was jointly funded by NSF and DARPA under NSF grant number IRI-8902124. space. A particular ranking of the hypotheses corre- sponds to a region (cell) defined by a set of inequalities that describe a polytope. In the hope of obtaining a more robust estimate, we measure the amount of slack for the different coefficients along the coordinate axes such that the weight remains within the cell as well as determine the "center" of the cell. The product of the slacks in the different coordinate directions at the "cen- ter" is an approximate indicator of the "volume" of the cell. If more than one cell gives the same performance, we choose the one with the largest "volume". Weights which correspond to the "center" of this cell are used for combining scores in the test set. 3. EXPERIMENTS Experiments were conducted to gain a better under- standing of the weight space. In our implementation of the N-Best rescoring paradigm [1], the N-Best list (N = 20) is generated by the BBN BYBLOS …

Content truncated to 500 words