AUTOMATIC MODEL REFINEMENT - with an application to tagging

AUTOMATIC MODEI~ REFINEMENT with an application to tagging Yi-Chung Lin, Tung-lIui Chiang and Keh-Yih Su Department of Electrical En,@Teering, National 7X'ing ltua University, Hsinchu, 7?liwan 300, Republic of China ABSTRACT Statistical NLP models usually only consider coarse information and very restricted context to make the estimation of parameters feasible. To reduce the modeling error introduced by a sim- plified probabilistic model, the Classitication and Regression Tree (CART) method was adopted in this paper to select more discriminative features for automatic model refinement. Because the features are adopted dependently during split- ting the classification tree in CART, the number of training data in each terminal node is small, which makes the labeling process of terminal nodes not robust. This over-tuning phenome- non cannot be completely removed by cross- validation process (i.e., pruning process). A probabilistic classification model based on the selected discriminative features is thtls proposed to use the training data more efficiently. In tag- ging the Brown Corpus, our probabilistic classi- fication model reduces the error rate of the top 10 error dominant words from 5.71% to 4.35%, which shows 23.82% improvement over the un- refined model. l. INTRODUCTION To automatically acquire knowledge from corpora, statistical methods are widely used re- cently (Church, 1989; Chiang, Lin & Su, 1992; Su, Chang & Lin, 1992). The perfonnance of a probabilistic model is affected by the es- timation error due to insufficient training data and the modeling error due to lacking complete knowledge of the problem to be conquered. In the literature, several smoothing methods (Good, 1953; Katz, 1987) have been used to effectively reduce the estimation error. On the contrary, the problem of reducing modeling error is less studied. Probabilistic models are usually simplified to make the estimation of parameters feasible. However, some important information may be lost while simplifying a model. For example, using the contextual words, instead of contextual parts of speech, enhances the prediction power for tagging parts of speech. But, unfortunately, reducing the m(×teling error by increasing the degree of model granularity is usually accompa- nied by a large estimation error if there is not enough training data. tIowever, if only the discriminative features arc involved (i.e., only those important param- eters are used), modeling error could be sig- niIicantly reduced without using a large co> pus. Those discriminative features usually vary for different words, and it would be very time- consuming to induce such features from the cor- pus manually. An algorithm for automatically extracting the discriminative features from a cor- pus is rims highly demanded. In this paper, the Classification and Regression Tree (CARl') method (Breiman, Friedman, Olshen & Stone, 1984) is first used to extract the discriminative features, l lowever, CAP, T basically regards all selected features as jointly dependent. Nodes in different branches are trained with different sets of data, and the available training data of a node becomes less and less while CART asks more and more questions. "FhereR)re, CART can easily split and prune the classification tree to fit the training …

Content truncated to 500 words