Low-Resource Formality Controlled NMT Using Pre-trained LM

Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), pages 321–329 July 13-14, 2023 c⃝2023 Association for Computational Linguistics Low-Resource Formality Controlled NMT Using Pre-trained LM Priyesh Vakharia and Shree Vignesh S and Pranjali Basmatkar Department of Computer Science University of California, Santa Cruz {pvakhari, ss64293, pbasmatk}@ucsc.edu Abstract This paper describes the UCSC’s submission to the shared task on formality control for spo- ken language translation at IWSLT 2023. For this task, we explored the use of “additive style intervention” using a pre-trained multilingual translation model, namely mBART. Compared to prior approaches where a single style-vector was added to all tokens in the encoder output, we explored an alternative approach in which we learn a unique style-vector for each input token. We believe this approach, which we call “style embedding intervention,” is better suited for formality control as it can potentially learn which specific input tokens to modify during decoding. While the proposed approach ob- tained similar performance to “additive style intervention” for the supervised English-to- Vietnamese task, it performed significantly bet- ter for English-to-Korean, in which it achieved an average matched accuracy of 90.6 compared to 85.2 for the baseline. When we constrained the model further to only perform style inter- vention on the <bos> (beginning of sentence) token, the average matched accuracy improved further to 92.0, indicating that the model could learn to control the formality of the translation output based solely on the embedding of the <bos> token. 1 Introduction In the past decade, neural machine translation has made remarkable strides, achieving transla- tion quality that is increasingly comparable to human-level performance across various languages. However, despite these advancements, the field of controllable machine translation remains rela- tively under-explored. One crucial aspect of transla- tion variation is formality, which manifests through grammatical registers, adapting the language to suit specific target audiences. Unfortunately, current neural machine translation (NMT) systems lack the capability to comprehend and adhere to gram- matical registers, specifically concerning formality. Consequently, this limitation can result in inaccu- racies in selecting the appropriate level of formal- ity, potentially leading to translations that may be deemed inappropriate in specific contexts. Recog- nizing the significance of formality control, we aim to build a formality-controlled machine translation system to foster smooth and reliable conversations and enhance communication across languages and cultures, facilitating more nuanced and effective linguistic exchanges. Formality-controlled Neural Machine Transla- tion is the IWSLT 2023 task (N˘adejde et al., 2022) under the Formality track. The goal of the task is to achieve formality controlled machine transla- tion for the English-Vietnamese (En-Vi), English- Korean (En-Ko) in a supervised setting and English- Portuguese (En-Pt) and English-Russian (En-Ru) in a zero-shot setting as detailed in (Agarwal et al., 2023). We provide an example of formal and infor- mal translations of an English sentence into Viet- namese in Figure 1. The formal and informal to- kens are in bold. 2 Related Works Machine translation (MT) research has primarily focused on preserving the meaning between lan- guages. However, it …

Content truncated to 500 words