Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9882 - 9902 December 7-11, 2022 ©2022 Association for Computational Linguistics Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature Katherine Thai⋆♢ Marzena Karpinska⋆♢ Kalpesh Krishna♢ William Ray♢ Moira Inghilleri♠ John Wieting♣ Mohit Iyyer♢ ♢Manning College of Information and Computer Sciences, UMass Amherst ♠Department of Languages, Literatures, and Cultures; UMass Amherst ♣Google Research {kbthai,mkarpinska,kalpesh,miyyer}@cs.umass.edu minghilleri@complit.umass.edu, jwieting@google.com Abstract Literary translation is a culturally significant task, but it is bottlenecked by the small number of qualified literary translators relative to the many untranslated works published around the world. Machine translation (MT) holds poten- tial to complement the work of human trans- lators by improving both training procedures and their overall efficiency. Literary translation is less constrained than more traditional MT settings since translators must balance mean- ing equivalence, readability, and critical inter- pretability in the target language. This prop- erty, along with the complex discourse-level context present in literary texts, also makes literary MT more challenging to computation- ally model and evaluate. To explore this task, we collect a dataset (PAR3) of non-English lan- guage novels in the public domain, each aligned at the paragraph level to both human and au- tomatic English translations. Using PAR3, we discover that expert literary translators prefer reference human translations over machine- translated paragraphs at a rate of 84%, while state-of-the-art automatic MT metrics do not correlate with those preferences. The experts note that MT outputs contain not only mistrans- lations, but also discourse-disrupting errors and stylistic inconsistencies. To address these prob- lems, we train a post-editing model whose out- put is preferred over normal MT output at a rate of 69% by experts. We publicly release PAR3 to spur future research into literary MT.1 1 Introduction While the quality of machine translation (MT) sys- tems has greatly improved with recent advances in modeling and dataset collection, the application of these new technologies to the task of automatically 1https://github.com/katherinethai/par3/ ⋆Authors contributed equally. translating literary text (e.g., novels, short stories) has remained limited to small-scale studies (Gen- zel et al., 2010; Jones and Irvine, 2013; Toral et al., 2018). Translating literary works differs from trans- lating standard MT corpora (e.g., news articles or parliamentary proceedings) in several key ways. For one, it is much more difficult to evaluate. The techniques2 used by literary translators differ fun- damentally from those applied in more standard MT domains (see Table 8 in the Appendix). Liter- ary translators have the freedom (or burden) of both semantic and critical interpretation, as they must solve the problem of equivalence, often beyond the word level (Neubert, 1983; Baker, 2018; Baker and Saldanha, 2021). The task of conveying an author’s ideas highlights yet another difference be- tween literary and traditional MT: document-level context is especially critical for the literary domain due to the presence of complex discourse structure, rendering the typical sentence-level MT pipeline insufficient for this task (Voigt and Jurafsky, 2012; Taivalkoski-Shilov, 2019). In this work, we seek to …

Content truncated to 500 words