Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3970–3979 August 1–6, 2021. ©2021 Association for Computational Linguistics 3970 Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation Tong Zhang1,2, Long Zhang1,2, Wei Ye1,†, Bo Li1,2, Jinan Sun1, Xiaoyu Zhu3, Wen Zhao1, Shikun Zhang1,† 1 National Engineering Research Center for Software Engineering, Peking University 2 School of Software and Microelectronics, Peking University 3 BIGO {zhangtong17,zhanglong418, wye, zhangsk}@pku.edu.cn Abstract This paper proposes a sophisticated neural ar- chitecture to incorporate bilingual dictionar- ies into Neural Machine Translation (NMT) models. By introducing three novel compo- nents: Pointer, Disambiguator, and Copier, our method PDC achieves the following mer- its inherently compared with previous efforts: (1) Pointer leverages the semantic information from bilingual dictionaries, for the first time, to better locate source words whose transla- tion in dictionaries can potentially be used; (2) Disambiguator synthesizes contextual in- formation from the source view and the target view, both of which contribute to distinguish- ing the proper translation of a specific source word from multiple candidates in dictionaries; (3) Copier systematically connects Pointer and Disambiguator based on a hierarchical copy mechanism seamlessly integrated with Trans- former, thereby building an end-to-end archi- tecture that could avoid error propagation prob- lems in alternative pipeline methods. The experimental results on Chinese-English and English-Japanese benchmarks demonstrate the PDC’s overall superiority and effectiveness of each component. 1 Introduction The past several years have witnessed the remark- able success of Neural machine translation (NMT), due to the development of sequence-to-sequence methods (Sutskever et al., 2014; Bahdanau et al., 2015; Vaswani et al., 2017). Since bilingual dic- tionaries cover rich prior knowledge, especially of low-frequency words, many efforts have been dedicated to incorporating bilingual dictionaries into NMT systems. These explorations can be roughly categorized into two broad paradigms. The first one transforms the bilingual dictionaries into pseudo parallel sentence pairs for training (Zhang †Corresponding authors. These patterns increase brake friction between tires and ground. 这些 花纹 可以 增强 轮胎 与 地面 之间 的 制动 摩擦 these pattern can increase tire and ground between of brake friction Source Input : Decoder Output : Bilingual Dictionary : Disambiguate rub friction clash conflict Copy Reference : These patterns increase brake Point 1 2 3 摩擦 : ？ Figure 1: Three key steps to translate with a bilin- gual dictionary: pointing, disambiguating and copying. This concrete illustrative example is chosen to conve- niently show the primary intuition behind our method. and Zong, 2016; Zhao et al., 2020). The second one utilizes the bilingual dictionaries as external re- sources fed into neural architectures (Luong et al., 2015; Gulcehre et al., 2016; Arthur et al., 2016; Zhang et al., 2017b; Zhao et al., 2018a,b, 2019b), which is more widely used and the focus of this paper. In practice, bilingual dictionaries usually contain more than one translation for a word. From a high- level perspective, we believe there are three criti- cal steps to incorporate …

Content truncated to 500 words