Controlling the Output Length of Neural Machine Translation

Controlling the Output Length of Neural Machine Translation Surafel Melaku Lakew⋆† Mattia Di Gangi⋆† Marcello Federico Amazon AI - Palo Alto, USA ⋆Fondazione Bruno Kessler, Trento - Italy † University of Trento, Italy Abstract The recent advances introduced by neural machine transla- tion (NMT) are rapidly expanding the application fields of machine translation, as well as reshaping the quality level to be targeted. In particular, if translations have to fit some given layout, quality should not only be measured in terms of adequacy and fluency, but also length. Exemplary cases are the translation of document files, subtitles, and scripts for dubbing, where the output length should ideally be as close as possible to the length of the input text. This pa- per addresses for the first time, to the best of our knowledge, the problem of controlling the output length in NMT. We in- vestigate two methods for biasing the output length with a transformer architecture: i) conditioning the output to a given target-source length-ratio class and ii) enriching the trans- former positional embedding with length information. Our experiments show that both methods can induce the network to generate shorter translations, as well as acquiring inter- pretable linguistic skills. 1. Introduction The sequence to sequence [1, 2] approach to Neural Machine Translation (NMT) has shown to improve quality in various translation tasks [3, 4, 5]. While translation quality is nor- mally measured in terms of correct transfer of meaning and of fluency, there are several applications of NMT that would benefit from optimizing the output length, such as the trans- lation of document elements that have to fit a given layout – e.g. entries of tables or bullet points of a presentation – or subtitles, which have to fit visual constraints and readability goals, as well as speech dubbing, for which the length of the translation should be as close as possible to the length of the original sentence. Current NMT models do not model explicitly sentence lengths of input and output, and the decoding methods do not allow to specify desired number of tokens to be generated. Instead, they implicitly rely on the observed length of the training examples [6, 7]. Sequence-to-sequence models have been also applied to text summarization [8] to map the relevant information found Authors are in random order and contributed equally. The first two authors carried out the work during their internship at Amazon. in a long text into a limited-length summary. Such models have shown promising results by directly controlling the out- put length [9, 10, 11, 12]. However, differently from MT, text summarization (besides being a monolingual task) is charac- terized by target sentences that are always much shorter than the corresponding source sentences. While in MT, the distri- bution of the relative lengths of source and target depends on the two languages and can significantly vary from one sen- tence pair to another due to stylistic decisions of the transla- tor and linguistic constraints (e.g. idiomatic expressions). In this work, we propose …

Content truncated to 500 words