Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1068–1077 August 1–6, 2021. ©2021 Association for Computational Linguistics 1068 Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation Shun-Po Chuang∗ Yung-Sung Chuang∗ Chih-Chiang Chang∗ Hung-yi Lee National Taiwan University, Taipei, Taiwan {f04942141,b05901033,r09922057,hungyilee}@ntu.edu.tw Abstract We study the possibilities of building a non-autoregressive speech-to-text translation model using connectionist temporal classifi- cation (CTC), and use CTC-based automatic speech recognition as an auxiliary task to im- prove the performance. CTC’s success on translation is counter-intuitive due to its mono- tonicity assumption, so we analyze its reorder- ing capability. Kendall’s tau distance is intro- duced as the quantitative metric, and gradient- based visualization provides an intuitive way to take a closer look into the model. Our anal- ysis shows that transformer encoders have the ability to change the word order and points out the future research direction that worth being explored more on non-autoregressive speech translation.1 1 Introduction Recently, there are more and more research works focusing on end-to-end speech translation (ST) (Bérard et al., 2016; Weiss et al., 2017; Bérard et al., 2018; Vila et al., 2018; Di Gangi et al., 2019; Ran et al., 2019; Chuang et al., 2020). Instead of cascading machine translation (MT) models to an automatic speech recognition (ASR) system, end- to-end models can skip the error bottleneck caused by ASR and be more computationally efficient. However, in the inference time, an autoregressive (AR) decoder is needed to decode the output se- quence, causing the latency issue. In MT, non-autoregressive (NAR) models have been heavily explored recently (Gu et al., 2018; Lee et al., 2018; Ghazvininejad et al., 2019; Stern et al., 2019; Gu et al., 2019; Saharia et al., 2020) by lever- aging the parallel nature of transformer (Vaswani et al., 2017). In contrast, such kind of models is rarely explored in the field of speech transla- tion, except for a concurrent work (Inaguma et al., ∗Contributed equally. 1The source code is available. See Appendix A. 2020). In this work, we use connectionist temporal classification (CTC) (Graves et al., 2006) to train NAR models for ST, without an explicit decoder module. Our entire model is merely a transformer encoder. Multitask learning (Anastasopoulos and Chiang, 2018; Kano et al., 2021) on ASR, which is often used in speech translation, can also be applied in our transformer encoder architecture to further push the performance. We achieve initial results on NAR speech translation by using a single speech encoder. CTC’s success on the translation task is counter- intuitive because of its monotonicity assumption. Previous works directly adopt the CTC loss on NAR translation without further verification on the reordering capability of CTC (Libovick`y and Helcl, 2018; Saharia et al., 2020; Inaguma et al., 2020). To further understand the reason that the CTC-based model can achieve ST task, we analyze the ordering capabilities of ST models by leveraging Kendall’s tau distance (Birch and Osborne, 2011; Kendall, 1938), and a gradient-based visualization is intro- duced to provide additional evidence. To the best …

Content truncated to 500 words