Controlling Machine Translation for Multiple Attributes with Additive Interventions

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6676–6696 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6676 Controlling Machine Translation for Multiple Attributes with Additive Interventions Andrea Schioppa∗and Artem Sokolov and David Vilar and Katja Filippova Google Research {arischioppa, artemsok, vilar, katjaf}@google.com Abstract Fine-grained control of machine transla- tion (MT) outputs along multiple attributes is critical for many modern MT applications and is a requirement for gaining users’ trust. standard approach for exerting control in MTA is to prepend the input with a special tag to signal the desired output attribute. Despite its simplicity, attribute tagging has several draw- backs: continuous values must be binned into discrete categories, which is unnatural for cer- tain applications; interference between multi- ple tags is poorly understood. We address these problems by introducing vector-valued interventions which allow for fine-grained con- trol over multiple attributes simultaneously via a weighted linear combination of the corre- sponding vectors. For some attributes, our approach even allows for fine-tuning a model trained without annotations to support such interventions. In experiments with three at- tributes (length, politeness and monotonicity) and two language pairs (English to German and Japanese) our models achieve better con- trol over a wider range of tasks compared to tagging, and translation quality does not de- grade when no control is requested. Finally, we demonstrate how to enable control in an already trained model after a relatively cheap fine-tuning stage. 1 Introduction Some modern machine translation (MT) applica- tions require fine-grained control along multiple attributes, and such mechanisms also increase the users’ trust in scenarios when the system speaks on their behalf (Prabhumoye et al., 2021). For example, MT applications like video subtitling in streaming, video conferencing, online education and speech MT require that one can control the length and monotonicity of the translation, setting clear constraints on the output. In open-domain ∗ Google AI Resident. MT, it is unlikely that such constraints are known or can be inferred from the source to generate an appropriate translation. However, the uncertainty around the desired register, style or politeness level of the translation could be resolved by providing users with an explicit option to control such at- tributes. This in turns increases the MT system’s trustworthiness by providing an explicit contract (Jacovi et al., 2021), formulated as “whenever there is an ambiguity, we enable users’ agency”. A standard method to exert control over MT outputs is the tagging approach, where an explicit token is prepended to the source sentence or out- put hypothesis to signal the desired attribute of the output (Kobus et al., 2017; Sennrich et al., 2016; Johnson et al., 2017). While such tags do enable certain level of control, discrete tags, by their na- ture, allow only for coarse-grained control and re- quire that attributes with continuous values, like monotonicity or length ratio, are binned. For ex- ample, Lakew et al. (2019) used only three tags to control translation length, which would arguably be too coarse for many practical applications. …

Content truncated to 500 words