Bilingual distributed phrase representations for statistical machin translation

Bilingual Distributed Phrase Representations for Statistical Machine Translation Peyman Passban ppassban@computing.dcu.ie Chris Hokamp chokamp@computing.dcu.ie Qun Liu qliu@computing.dcu.ie ADAPT Centre, School of Computing, Dublin City University, Ireland Abstract Phrase–based machine translation (PBMT) relies upon the phrase-table as the main resource for bilingual knowledge at decoding time. A phrase table in its basic form includes aligned phrases along with four probabilities indicating aspects of the co-occurrence statistics for each phrase pair. In this paper we add a new semantic similarity score as a statistical feature to enrich the phrase table. The new feature is inferred from a bilingual corpus by a neural net- work (NN), and estimates the semantic similarity of each source and target phrase pair. We observe a significant increase in system performance with the addition of the new feature. We evaluated our model on the English–French (En–Fr) and English–Farsi (En–Fa) language pairs. Experimental results show improvements for all translation directions of En↔Fr and En↔Fa. 1 Introduction Phrase–based machine translation begins by segmenting a source sentence into phrases and looking up candidate translations for the sub-sentential phrases in a phrase table. The goal of PBMT is to assemble these translation candidates into the optimal target sequence. Finding the best source segmentation, translation candidates, and phrase ordering is a search problem which is typically formulated as a log-linear model using both dynamic and static features, whose weights are optimized via a search heuristic on held-out development data. The inverse phrase translation probability ϕ(f|e), inverse lexical weighting lex(f|e), direct phrase trans- lation probability ϕ(e|f) and direct lexical weighting lex(e|f) are four of the standard static features used in phrase tables. These features are computed directly from the co-occurrence of aligned phrases in the training corpora. However, co-occurrence information alone cannot capture semantic information about phrases, especially when they are taken out of context. Therefore, many techniques have been proposed to enrich the feature list by including features which contain syntactic and/or semantic information (Banchs and Costa-juss`a, 2011). Most work evaluating the inclusion of semantic information into SMT decoders has fo- cused upon adding dynamic features (those which must be computed at decoding time). How- ever, dynamic features require the implementation of a new feature function which depends upon the hypothesis data structure of the particular decoder. The implementation of dynamic features typically requires significantly more engineering effort than simply augmenting the phrase table. In this paper, we add a new static feature and show how a good static feature alone can significantly boost translation quality. The basic idea behind our work is to use the vector representation, or semantic embedding of phrases, which is generated by an NN. The semantic features of the source and target languages are projected into a shared bilingual space, Proceedings of MT Summit XV, vol.1: MT Researchers' Track Miami, Oct 30 - Nov 3, 2015 | p. 310 which preserves both semantic and syntactic information about the phrases. The supervised approach to generating embeddings from neural networks optimizes the network to produce vectors which are good with …

Content truncated to 500 words