A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 109–118, Osaka, Japan, December 11-17 2016. A Correlational Encoder Decoder Architecture for Pivot Based Sequence Generation Amrita Saha IBM Research India amrsaha4@in.ibm.com Janarthanan Rajendran I.I.T. Madras, India rsdjjana@gmail.com Mitesh M. Khapra I.I.T. Madras, India khapra.mitesh@gmail.com Sarath Chandar University of Montreal anbilpas@iro.umontreal.ca Kyunghyun Cho New York University cho.k.hyun@gmail.com Abstract Interlingua based Machine Translation (MT) aims to encode multiple languages into a common linguistic representation and then decode sentences in multiple target languages from this repre- sentation. In this work we explore this idea in the context of neural encoder decoder architectures, albeit on a smaller scale and without MT as the end goal. Specifically, we consider the case of three languages or modalities X, Z and Y wherein we are interested in generating sequences in Y starting from information available in X. However, there is no parallel training data available between X and Y but, training data is available between X & Z and Z & Y (as is often the case in many real world applications). Z thus acts as a pivot/bridge. An obvious solution, which is perhaps less elegant but works very well in practice is to train a two stage model which first converts from X to Z and then from Z to Y . Instead we explore an interlingua inspired solu- tion which jointly learns to do the following (i) encode X and Z to a common representation and (ii) decode Y from this common representation. We evaluate our model on two tasks: (i) bridge transliteration and (ii) bridge captioning. We report promising results in both these ap- plications and believe that this is a right step towards truly interlingua inspired encoder decoder architectures. 1 Introduction Interlingua based MT (Nirenburg, 1994; Dorr et al., 2010) relies on the principle that every language in the world can be mapped to a common linguistic representation. Further, given this representation, it should be possible to decode a target sentence in any language. This implies that given n languages we just need n decoders and n encoders to translate between these nC2 language pairs. Note that even though we take inspiration from interlingua based MT, the focus of this work is not on MT. We believe that this idea is not just limited to translation but could be applicable to any kind of conversion involving multiple source and target languages and/or modalities (for example, transliteration, multilingual image captioning, multilingual image Question Answering, etc.). Even though this idea has had limited success, it is still fascinating and considered by many as the holy grail of multilingual multimodal processing. It is interesting to consider the implications of this idea when viewed in the statistical context. For example, current state of the art statistical systems for MT (Koehn et al., 2003; Chiang, 2005; Luong et al., 2015b), transliteration (Finch et al., 2015; Shao et al., 2015; Nicolai et al., 2015), image captioning (Vinyals et al., 2015b; Xu et al., 2015), …

Content truncated to 500 words