Multi-agent Learning for Neural Machine Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 856–865, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 856 Multi-agent Learning for Neural Machine Translation Tianchi Bi, Hao Xiong, Zhongjun He, Hua Wu and Haifeng Wang Baidu Inc. No. 10, Shangdi 10th Street Beijing, 100085, China {bitianchi, xionghao05, hezhongjun, wu hua, wanghaifeng}@baidu.com Abstract Conventional Neural Machine Translation (NMT) models benefit from the training with an additional agent, e.g., dual learning, and bidirectional decoding with one agent decod- ing from left to right and the other decoding in the opposite direction. In this paper, we extend the training framework to the multi-agent sce- nario by introducing diverse agents in an in- teractive updating process. At training time, each agent learns advanced knowledge from others, and they work together to improve translation quality. Experimental results on NIST Chinese-English, IWSLT 2014 German- English, WMT 2014 English-German and large-scale Chinese-English translation tasks indicate that our approach achieves absolute improvements over the strong baseline sys- tems and shows competitive performance on all tasks. 1 Introduction Training with more than one agents has attracted intensive research interest in recent years, for ex- ample, dual learning (He et al., 2016; Xia et al., 2017, 2018) and bidirectional decoding (Liu et al., 2016; Zhang et al., 2019b). The former method leverages the duality between the two related agents as the feedback signal to regularize train- ing, while the latter targets the agreement between one agent decoding from left to right (L2R) while the other decoding in opposite direction (R2L). Both methods enhance the translation models by introducing a regularization term into the training objective. The effectiveness of these two methods lies in the fact that appropriate regularization can help each agent to learn from the superior models while integrating their advantages (e.g., good transla- tions for prefixes for L2R, and good translation quality for suffixes for R2L). As shown in the Ta- ble 1, due to the exposure bias problem (Ranzato Src 此次 豪雨 灾情 , 半数 遇难 者 遭 洪水 冲走 溺 毙 ; 其他 则 因 房屋 倒塌 或 电线 走火 触电 致死 。 Ref In this disaster caused by torrential rains, half of the victims were carried away and drowned in the floods; the others died because their houses collapsed or from electrocution caused by short circuits. L2R In the torrential rain, half of the vic- tims were washed away by floods, while others died of electricity caused by col- lapsed houses or electric wires. R2L Half of the victims were drowned by floods, while others were killed by collapsed houses or electrocuted by sparkling wires. Table 1: In this sample, NMT systems with different implementations present diverse translation errors for one long sentence. The agent decoding from left to right (L2R) tends to generate better prefixes and bad suffixes (bold letters), and the R2L agent has the oppo- site preference. et al., 2016), the agent trained to …

Content truncated to 500 words