Don’t Go Far Off: An Empirical Study on Neural Poetry Translation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7253–7265 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 7253 Don’t Go Far Off: An Empirical Study on Neural Poetry Translation Tuhin Chakrabarty 1, Arkadiy Saakyan1, and Smaranda Muresan1,2 1Department of Computer Science, Columbia University 2Data Science Institute, Columbia University tuhin.chakr@cs.columbia.edu, {a.saakyan,smara}@columbia.edu Abstract Despite constant improvements in machine translation quality, automatic poetry transla- tion remains a challenging problem due to the lack of open-sourced parallel poetic corpora, and to the intrinsic complexities involved in preserving the semantics, style and figurative nature of poetry. We present an empirical investigation for poetry translation along sev- eral dimensions: 1) size and style of training data (poetic vs. non-poetic), including a zero- shot setup; 2) bilingual vs. multilingual learn- ing; and 3) language-family-specific models vs. mixed-language-family models. To accom- plish this, we contribute a parallel dataset of poetry translations for several language pairs. Our results show that multilingual fine-tuning on poetic text significantly outperforms multi- lingual fine-tuning on non-poetic text that is 35X larger in size, both in terms of automatic metrics (BLEU, BERTScore, COMET) and human evaluation metrics such as faithfulness (meaning and poetic style). Moreover, multi- lingual fine-tuning on poetic data outperforms bilingual fine-tuning on poetic data. 1 1 Introduction American poet Robert Frost once defined poetry as “that which gets lost out of both prose and verse in translation” (Frost, 1961). Indeed, the task is so complex that translators often have to “create a poem in the target language which is readable and enjoyable as an independent, literary text" (Jones, 2011). But even though poetry is destined to lose its accuracy, integrity, and beauty even in human translation, the process conceives new opportuni- ties to stress-test the ability of machine translation models to deal with figurative language. While most computational work has focused on poetry generation (Hopkins and Kiela, 2017; Uthus 1The italics part of the title is the translation of a poem by Pablo Neruda with the same name. 2Example taken from (Ghazvininejad et al., 2018). Original Il ny avait que sable et boue Où sétait ouverte la tombe. Le long des murs de la prison On ne voyait aucune tombe Human For where a grave had opened wide, There was no grave at all: Only a stretch of mud and sand By the hideous prison-wall, GCK But there was only sand and mud. To where the grave was laid. Along the walls of prison wall. We saw no masquerade Google There was only sand and mud Where the grave had opened. Along the prison walls No tomb could be seen. Table 1: A French poem accompanied by human translation, (Ghazvininejad et al., 2018) (GCK) system translation, and Google Translate.2 et al., 2021; Van de Cruys, 2020; Ghazvininejad et al., 2016; Li et al., 2020; Hämäläinen and Alnaj- jar, 2019; Yi et al., 2020; Deng et al., 2019; Chen et al., 2019; Yang et al., 2018), research on poetry translation is in its infancy (Ghazvininejad et al., …

Content truncated to 500 words