Kernel Regression Framework for Machine Translation: UCL System Description for WMT 2008 Shared Translation Task

Proceedings of the Third Workshop on Statistical Machine Translation, pages 155–158, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Kernel Regression Framework for Machine Translation: UCL System Description for WMT 2008 Shared Translation Task Zhuoran Wang University College London Dept. of Computer Science Gower Street, London, WC1E 6BT United Kingdom z.wang@cs.ucl.ac.uk John Shawe-Taylor University College London Dept. of Computer Science Gower Street, London, WC1E 6BT United Kingdom jst@cs.ucl.ac.uk Abstract The novel kernel regression model for SMT only demonstrated encouraging results on small-scale toy data sets in previous works due to the complexities of kernel methods. It is the first time results based on the real-world data from the shared translation task will be reported at ACL 2008 Workshop on Statisti- cal Machine Translation. This paper presents the key modules of our system, including the kernel ridge regression model, retrieval-based sparse approximation, the decoding algorithm, as well as language modeling issues under this framework. 1 Introduction This paper follows the work in (Wang et al., 2007; Wang and Shawe-Taylor, 2008) which applied the kernel regression method with high-dimensional outputs proposed originally in (Cortes et al., 2005) to statistical machine translation (SMT) tasks. In our approach, the machine translation problem is viewed as a string-to-string mapping, where both the source and the target strings are embedded into their re- spective kernel induced feature spaces. Then ker- nel ridge regression is employed to learn the map- ping from the input feature space to the output one. As a kernel method, this model offers the potential advantages of capturing very high-dimensional cor- respondences among the features of the source and target languages as well as easy integration of ad- ditional linguistic knowledge via selecting particu- lar kernels. However, unlike the sequence labeling tasks such as optical character recognition in (Cortes et al., 2005), the complexity of the SMT problem it- self together with the computational complexities of kernel methods significantly complicate the imple- mentation of the regression technique in this field. Our system is actually designed as a hybrid of the classic phrase-based SMT model (Koehn et al., 2003) and the kernel regression model as follows: First, for each source sentence a small relevant set of sentence pairs are retrieved from the large-scale par- allel corpus. Then, the regression model is trained on this small relevant set only as a sparse approx- imation of the regression hyperplane trained on the entire training set, as proposed in (Wang and Shawe- Taylor, 2008). Finally, a beam search algorithm is utilized to decode the target sentence from the very noisy output feature vector we predicted, with the support of a pre-trained phrase table to generate pos- sible hypotheses (candidate translations). In addi- tion, a language model trained on a monolingual cor- pus can be integrated either directly into the regres- sion model or during the decoding procedure as an extra scoring function. Before describing each key component of our sys- tem in detail, we give a block diagram overview in Figure 1. 2 Problem Formulation …

Content truncated to 500 words