Enriching Phrase Tables for Statistical Machine Translation Using Mixed Embeddings

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2582–2591, Osaka, Japan, December 11-17 2016. Enriching Phrase Tables for Statistical Machine Translation Using Mixed Embeddings Peyman Passban, Qun Liu and Andy Way ADAPT Centre School of Computing Dublin City University, Ireland firstname.lastname@adaptcentre.ie Abstract The phrase table is considered to be the main bilingual resource for the phrase-based statisti- cal machine translation (PBSMT) model. During translation, a source sentence is decomposed into several phrases. The best match of each source phrase is selected among several target-side counterparts within the phrase table, and processed by the decoder to generate a sentence-level translation. The best match is chosen according to several factors, including a set of bilingual features. PBSMT engines by default provide four probability scores in phrase tables which are considered as the main set of bilingual features. Our goal is to enrich that set of features, as a better feature set should yield better translations. We propose new scores generated by a Con- volutional Neural Network (CNN) which indicate the semantic relatedness of phrase pairs. We evaluate our model in different experimental settings with different language pairs. We observe significant improvements when the proposed features are incorporated into the PBSMT pipeline. 1 Introduction PBSMT models sentence-level translation with a phrase-based setting in which sentences are decom- posed into different phrases (Koehn et al., 2007; Koehn, 2009). At each step, for a given source phrase the best candidate among the target phrases is selected as its translation. Phrasal translations are com- bined together to produce the sentence-level translation. This is a high-level view of PBSMT and there are many other processes involved in the main pipeline. Different bilingual and monolingual features are taken into account to make the final translation as adequate and fluent as possible. In this paper we only focus on the phrase-pairing process and try to enrich that part. The standard baseline bilingual features in the PBSMT pipeline by default are: the phrase translation probability φ(e|f), inverse phrase translation probability φ(f|e), lexical weighting lex(e|f) and inverse lexical weighting lex(f|e). These scores are computed based on the co-occurrence of phrase pairs in training corpora and do not indicate any other information about phrases, their relation or context. Our goal in this research is to extend this set of features by incorporating semantic information of phrase pairs. Word embeddings are numerical representations of words which preserve semantic and syntactic in- formation about words themselves and their context (Huang et al., 2012; Luong et al., 2013; Mikolov et al., 2013a; Mikolov et al., 2013b). They also preserve information about word order. These types of contextual, syntactic and semantic information can be quite useful for machine translation (MT), but being by its very nature a bilingual application, it requires bilingual (cross-lingual) information. Accord- ingly, we need methods to train bilingual embeddings via which we can access syntactic and semantic information about the source side as well as the target. Several papers have explored the training of bilingual embeddings …

Content truncated to 500 words