Project PiPeNovel: Pilot on Post-editing Novels

Project PiPeNovel: Pilot on Post-editing Novels Antonio Toral, Martijn Wieling Center for Language and Cognition Faculty of Arts University of Groningen, The Netherlands {a.toral.ruiz,m.b.wieling}@rug.nl Sheila Castilho, Joss Moorkens, Andy Way ADAPT Centre School of Computing Dublin City University, Ireland firstname.secondname@adaptcentre.ie Abstract Given (i) the rise of a new paradigm to machine translation based on neural net- works that results in more fluent and less literal output than previous models and (ii) the maturity of machine-assisted translation via post-editing in industry, project PiPeNovel studies the feasibility of the post-editing workflow for literary text conducting experiments with profes- sional literary translators. Machine translation (MT) has progressed enor- mously over the last years and it is widely used nowadays for gisting purposes. However, its use in professional translation is still largely confined to the post-editing of technical and legislative text. The aim of PiPeNovel is to carry out a pilot study to assess the feasibility of broadening the use of the post-editing workflow to literary text, in particular to novels. The translation direction covered in the project is English-to-Catalan. Now PiPeNovel is about to finish and we present the three main activities conducted in the project: (1) MT. First, we built a literary-adapted neu- ral MT (NMT) system and evaluated it against a system pertaining to the previous dominant para- digm in MT: statistical phrase-based MT (PB- SMT) (Toral and Way, 2018). Both systems were trained on over 1,000 novels. We conducted a hu- man evaluation on three novels by Orwell, Rowl- ing and Salinger; between 17% and 34% of the translations, depending on the book, produced by NMT (versus 8% and 20% with PBSMT) were perceived by native speakers of the target lan- guage to be of equivalent quality to translations produced by a professional human translator. (2) Post-editing effort. Subsequently, using these MT systems, we conducted a post-editing © 2018 The authors. This article is licensed under a Cre- ative Commons 3.0 licence, no derivative works, attribution, CC-BY-ND. study with six professional literary translators on a fantasy novel (Toral et al., 2018). We analysed temporal effort and found that both MT ap- proaches result in increases in translation produc- tivity: PBMT by 18%, and NMT by 36%. Post- editing also led to reductions in the number of keystrokes (technical effort): by 9% with PBMT, and by 23% with NMT. Finally, regarding cogni- tive effort, post-editing resulted in fewer (29% and 42% less with PBMT and NMT respectively) but longer pauses (14% and 25%). (3) Translators’ perceptions. Finally, we ana- lysed the perceptions of the translators that took part in the post-editing experiment (Moorkens et al., 2018), which were collected via question- naires and a debrief session. While, as stated be- fore, all participants were faster when post-edit- ing NMT, they all still stated a preference for translation from scratch, as they felt less con- strained and could be more creative. When com- paring MT systems, participants found NMT out- put to be more fluent and adequate. Acknowledgements PiPeNovel is funded by …

Content truncated to 500 words