L1 Regularized Regression for Reranking and System Combination in Machine Translation

Proceedings of the Joint 5th Workshop on Statistical Machine Translation and MetricsMATR, pages 282–289, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics L1 Regularized Regression for Reranking and System Combination in Machine Translation Ergun Bic¸ici Koc¸ University 34450 Sariyer, Istanbul, Turkey ebicici@ku.edu.tr Deniz Yuret Koc¸ University 34450 Sariyer, Istanbul, Turkey dyuret@ku.edu.tr Abstract We use L1 regularized transductive regres- sion to learn mappings between source and target features of the training sets derived for each test sentence and use these mappings to rerank translation out- puts. We compare the effectiveness of L1 regularization techniques for regression to learn mappings between features given in a sparse feature matrix. The results show the effectiveness of using L1 regulariza- tion versus L2 used in ridge regression. We show that regression mapping is ef- fective in reranking translation outputs and in selecting the best system combinations with encouraging results on different lan- guage pairs. 1 Introduction Regression can be used to find mappings be- tween the source and target feature sets derived from given parallel corpora. Transduction learn- ing uses a subset of the training examples that are closely related to the test set without using the model induced by the full training set. In the context of SMT, we select a few training in- stances for each test instance to guide the transla- tion process. This also gives us a computational advantage when considering the high dimension- ality of the problem. The goal in transductive regression based machine translation (TRegMT) is both reducing the computational burden of the regression approach by reducing the dimension- ality of the training set and the feature set and also improving the translation quality by using transduction. Transductive regression is shown to achieve higher accuracy than L2 regularized ridge regression on some machine learning benchmark datasets (Chapelle et al., 1999). In an idealized feature mapping matrix where features are word sequences, we would like to ob- serve few target features for each source feature derived from a source sentence. In this setting, we can think of feature mappings being close to per- mutation matrices with one nonzero item for each column. L1 regularization helps us achieve solu- tions close to the permutation matrices by increas- ing sparsity. We show that L1 regularized regression map- ping is effective in reranking translation outputs and present encouraging results on different lan- guage pairs in the translation task of WMT10. In the system combination task, different translation outputs of different translation systems are com- bined to find a better translation. We model system combination task as a reranking problem among the competing translation models and present en- couraging results with the TRegMT system. Related Work: Regression techniques can be used to model the relationship between strings (Cortes et al., 2007). Wang et al. (2007) applies a string-to-string mapping approach to machine translation by using ordinary least squares regression and n-gram string kernels to a small dataset. Later they use L2 regularized least squares regression (Wang and Shawe-Taylor, 2008). Although the translation …

Content truncated to 500 words