Post-editing in Automatic Subtitling: A Subtitlers’ perspective

Neural Speech Translation: from Neural Machine Translation to Direct Speech translation Mattia A. Di Gangi∗ Fondazione Bruno Kessler (FBK) ICT Doctoral School - University of Trento via Sommarive, Povo, Trento, Italy mattia.digangi@unitn.it Speech-to-text translation, or simply speech translation (ST), is the task of translating automati- cally a spoken speech. The problem has classically been tackled by combining the technologies of au- tomatic speech recognition (ASR) and machine translation (MT) with different degrees of coupling (Takezawa et al., 1998; Waibel et al., 1991). The most popular approach is to cascade ASR and MT systems, as it can make use of the state of the art in such mature fields (Black et al., 2002). The goal of this thesis was to develop the so-called approach of direct speech translation, which translates au- dio without intermediate transcription (Duong et al., 2016; B´erard et al., 2016; Weiss et al., 2017). Direct speech translation (DST) is based on the sequence-to-sequence learning technology that al- lowed the spectacular advances of the field of neu- ral MT (NMT) but introducing its own challenges (Sutskever et al., 2014; Bahdanau et al., 2015). We started with a study about the effects of NMT in cascaded ST, where we analyzed the translation errors of NMT and phrase-based MT (PBMT) for automatically transcribed input text. Our results showed that NMT achieves an overall higher quality also in this setting, but its ability to model a theoretically-unlimited context can intro- duce subtle errors. Indeed, we found that in PBMT the errors are localized in correspondence to the source error, whereas NMT can introduce errors far from the source-side error position. Motivated by application needs, in a following work we studied how to use a single NMT system to translate effectively clean source text and auto- matic transcripts. We found that a simple training ∗*Now at AppTek GmbH ∗© 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. algorithm that fine-tunes the model on both kinds of inputs improves the translation quality of cor- rupted input without any degradation on clean in- put. In a parallel research line, we were interested in making the training of RNN-based NMT more efficient, as it required at the time long training time also for relatively small datasets. For this, we proposed simple-recurrent NMT (SR-NMT), an encoder-decoder architecture that requires a fraction of parameters and computing power than LSTM-based NMT. It is built on top of simple re- current units (Lei et al., 2017), which are faster to train but achieve a lower translation quality than LSTMs, particularly because they do not bene- fit from the addition of computation layers. On the other side, SR-NMT has been designed to be trained as a deep network and our results show how the performance improves significantly up to 8 lay- ers in the encoder and in the decoder. Our two research lines converge in our work on DST. We start with a participation in IWSLT 2018, which introduced a …

Content truncated to 500 words