Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5777 - 5790 July 10-15, 2022 ©2022 Association for Computational Linguistics Neighbors Are Not Strangers: Improving Non-Autoregressive Translation under Low-Frequency Lexical Constraints Chun Zeng♠∗, Jiangjie Chen♠∗, Tianyi Zhuang♠, Rui Xu♠, Hao Yang♢, Ying Qin♢, Shimin Tao♢, Yanghua Xiao♠♣† ♠Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University ♢Huawei Translation Services Center ♣Fudan-Aishu Cognitive Intelligence Joint Research Center {zengc20,jjchen19,tyzhuang20,shawyh}@fudan.edu.cn, ruixu21@m.fudan.edu.cn, {yanghao30,qinying,taoshimin}@huawei.com Abstract Lexically constrained neural machine transla- tion (NMT) draws much industrial attention for its practical usage in specific domains. How- ever, current autoregressive approaches suffer from high latency. In this paper, we focus on non-autoregressive translation (NAT) for this problem for its efficiency advantage. We identify that current constrained NAT models, which are based on iterative editing, do not handle low-frequency constraints well. To this end, we propose a plug-in algorithm for this line of work, i.e., Aligned Constrained Training (ACT), which alleviates this problem by famil- iarizing the model with the source-side context of the constraints. Experiments on the gen- eral and domain datasets show that our model improves over the backbone constrained NAT model in constraint preservation and translation quality, especially for rare constraints.1 1 Introduction Despite the success of neural machine translation (NMT) (Bahdanau et al., 2015; Vaswani et al., 2017; Barrault et al., 2020), real applications usu- ally require the precise (if not exact) translation of specific terms. One popular solution is to incor- porate dictionaries of pre-defined terminologies as lexical constraints to ensure the correct translation of terms, which has been demonstrated to be ef- fective in many areas such as domain adaptation, interactive translation, etc. Previous methods on lexically constrained translation are mainly built upon Autoregressive Translation (AT) models, imposing constraints at inference-time (Ture et al., 2012; Hokamp and Liu, ∗Authors contributed equally. †Corresponding author. 1Our implementation can be found at https:// github.com/sted-byte/ACT4NAT. Source �Travellers �� � 1.8K �screamed �� � 24 and ���� 2.8M �children �� 30.0K� cried � �� � 122 . Target �Reisende �� � 944 h¨atten � �� � 9.9K �geschrien �� � 13 und ���� 2.6M Kinder � �� � 20.1K geweint � �� � 13 . Terminology Constraints scream → geschrien Unconstrained translation Reisende schrien und Kinder rieen. ⇒ wrong term Soft constrained translation Reisende rien. ⇒ incomplete sentence & wrong term Hard constrained translation Reisende geschrien. ⇒ incomplete sentence Table 1: Translation examples of a lexically constrained non-autoregressive translation (NAT) model (Gu et al., 2019) under a low-frequency word as constraint. The underbraced word frequencies (uncased) are calculated from the vast WMT14 English-German translation (En- De) datasets (Vaswani et al., 2017). 2017; Post and Vilar, 2018) or training-time (Lu- ong et al., 2015; Ailem et al., 2021). However, such methods either are time-consuming in real-time ap- plications or do not ensure the appearance of con- straints in the output. To develop faster MT mod- els for industrial applications, Non-Autoregressive Translation (NAT) has been put …

Content truncated to 500 words