EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 8141 - 8153 May 22-27, 2022 c⃝2022 Association for Computational Linguistics EAG: Extract and Generate Multi-way Aligned Corpus for Complete Multi-lingual Neural Machine Translation Yulin Xu1∗, Zhen Yang1∗, Fandong Meng1, and Jie Zhou1 1Pattern Recognition Center, WeChat AI, Tencent Inc, China {xuyulincs}@gmail.com {zieenyang,fandongmeng,withtomzhou}@tencent.com Abstract Complete Multi-lingual Neural Machine Translation (C-MNMT) achieves superior performance against the conventional MNMT by constructing multi-way aligned corpus, i.e., aligning bilingual training examples from different language pairs when either their source or target sides are identical. However, since exactly identical sentences from differ- ent language pairs are scarce, the power of the multi-way aligned corpus is limited by its scale. To handle this problem, this paper proposes "Extract and Generate" (EAG), a two-step approach to construct large-scale and high-quality multi-way aligned corpus from bilingual data. Specifically, we first extract candidate aligned examples by pairing the bilingual examples from different language pairs with highly similar source or target sentences; and then generate the final aligned examples from the candidates with a well- trained generation model. With this two-step pipeline, EAG can construct a large-scale and multi-way aligned corpus whose diversity is almost identical to the original bilingual corpus. Experiments on two publicly available datasets i.e., WMT-5 and OPUS-100, show that the proposed method achieves significant improvements over strong baselines, with +1.1 and +1.4 BLEU points improvements on the two datasets respectively. 1 Introduction Multilingual Neural Machine Translation (MMMT) (Dong et al., 2015; Firat et al., 2017; Johnson et al., 2017; Aharoni et al., 2019) has achieved promis- ing results on serving translations between multi- ple language pairs with one model. With sharing parameters of the model, MNMT can facilitate in- formation sharing between similar languages and make it possible to translate between low-resource ∗Equal contribution. Work was done when Yulin Xu was interning at Pattern Recognition Center, WeChat AI, Tencent Inc, China. and zero-shot language pairs. Since the majority of available MT training data are English-centric, i.e., English either as the source or target language, most non-English language pairs do not see a sin- gle training example when training MNMT models (Freitag and Firat, 2020). Therefore, the perfor- mance of MNMT models on non-English trans- lation directions still left much to be desired: 1) Lack of training data leads to lower performance for non-English language pairs(Zhang et al., 2021); 2) MNMT models cannot beat the pivot-based base- line systems which translate non-English language pairs by bridging through English (Cheng et al., 2016; Habash and Hu, 2009). Recently, Freitag and Firat (2020) re-kindle the flame by proposing C-MNMT, which trains the model on the constructed multi-way aligned corpus. Specifically, they extract the multi-way aligned ex- amples by aligning training examples from differ- ent language pairs when either their source or target sides are identical (i.e., pivoting through English, for German→English and English→French to ex- tract German-French-English examples). Since they directly extract the multi-way aligned exam- ples from the bilingual corpus, we …

Content truncated to 500 words