Tree-to-Sequence Attentional Neural Machine Translation

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 823–833, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Tree-to-Sequence Attentional Neural Machine Translation Akiko Eriguchi, Kazuma Hashimoto, and Yoshimasa Tsuruoka The University of Tokyo, 3-7-1 Hongo, Bunkyo-ku, Tokyo, Japan {eriguchi, hassy, tsuruoka}@logos.t.u-tokyo.ac.jp Abstract Most of the existing Neural Machine Translation (NMT) models focus on the conversion of sequential data and do not directly use syntactic information. We propose a novel end-to-end syntac- tic NMT model, extending a sequence- to-sequence model with the source-side phrase structure. Our model has an at- tention mechanism that enables the de- coder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. Experi- mental results on the WAT’15 English- to-Japanese dataset demonstrate that our proposed model considerably outperforms sequence-to-sequence attentional NMT models and compares favorably with the state-of-the-art tree-to-string SMT system. 1 Introduction Machine Translation (MT) has traditionally been one of the most complex language processing problems, but recent advances of Neural Machine Translation (NMT) make it possible to perform translation using a simple end-to-end architecture. In the Encoder-Decoder model (Cho et al., 2014b; Sutskever et al., 2014), a Recurrent Neural Net- work (RNN) called the encoder reads the whole sequence of source words to produce a fixed- length vector, and then another RNN called the decoder generates the target words from the vec- tor. The Encoder-Decoder model has been ex- tended with an attention mechanism (Bahdanau et al., 2015; Luong et al., 2015a), which allows the model to jointly learn the soft alignment between the source language and the target language. NMT models have achieved state-of-the-art results in English-to-French and English-to-German trans- Figure 1: Alignment between an English phrase and a Japanese word. lation tasks (Luong et al., 2015b; Luong et al., 2015a). However, it is yet to be seen whether NMT is competitive with traditional Statistical Machine Translation (SMT) approaches in trans- lation tasks for structurally distant language pairs such as English-to-Japanese. Figure 1 shows a pair of parallel sentences in English and Japanese. English and Japanese are linguistically distant in many respects; they have different syntactic constructions, and words and phrases are defined in different lexical units. In this example, the Japanese word “緑茶” is aligned with the English words “green” and “tea”, and the English word sequence “a cup of” is aligned with a special symbol “null”, which is not explic- itly translated into any Japanese words. One way to solve this mismatch problem is to consider the phrase structure of the English sentence and align the phrase “a cup of green tea” with “緑茶”. In SMT, it is known that incorporating syntactic con- stituents of the source language into the models improves word alignment (Yamada and Knight, 2001) and translation accuracy (Liu et al., 2006; Neubig and Duh, 2014). However, the existing NMT models do not allow us to perform this kind of alignment. In this paper, we propose a novel attentional NMT model to …

Content truncated to 500 words