GupShup: Summarizing Open-Domain Code-Switched Conversations

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 39–55 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 39 Cross Attention Augmented Transducer Networks for Simultaneous Translation Dan Liu1,2, Mengge Du2, Xiaoxi Li2, Ya Li2 and Enhong Chen 1 1University of Science and Technology of China, Hefei, China 2iFlytek Research, Hefei, China danliu@mail.ustc.edu.cn , cheneh@ustc.edu.cn {danliu,mgdou,xxli8,yali8}@iflytek.com Abstract This paper proposes a novel architecture, Cross Attention Augmented Transducer (CAAT), for simultaneous translation. The framework aims to jointly optimize the policy and translation models. To effectively consider all possible READ-WRITE simul- taneous translation action paths, we adapt the online automatic speech recognition (ASR) model, RNN-T, but remove the strong monotonic constraint, which is critical for the translation task to consider reordering. To make CAAT work, we introduce a novel latency loss whose expectation can be opti- mized by a forward-backward algorithm. We implement CAAT with Transformer while the general CAAT architecture can also be implemented with other attention-based encoder-decoder frameworks. Experiments on both speech-to-text (S2T) and text-to-text (T2T) simultaneous translation tasks show that CAAT achieves significantly better latency-quality trade-offs compared to the state-of-the-art simultaneous translation approaches. 1 1 Introduction Simultaneous translation, which starts to translate input sentences before they are finished, is of im- portance to many real-life applications such as tele- conference systems and time-sensitive spoken doc- ument analysis and conversion. While a substantial progress has been made on offline machine transla- tion (Wu et al., 2016; Vaswani et al., 2017; Hassan et al., 2018), more research on simultaneous trans- lation is yet highly desirable. Central to the task is performing high-quality low-latency translation, which involves the key challenges of developing op- timal policies for the READ-WRITE action paths 1The code is available at https://github.com/ danliu2/caat. as well as generating high-quality target sequences based only on partial source sequences. This paper aims to optimize the policy and translation model jointly, by expanding target se- quences with blank symbols for READ actions. The loss function can be defined as negative log- likelihood (NLL) of marginal distribution through all expanded paths. A similar problem in automatic speech recognition (ASR) has been tackled with RNN-T (Recurrent Neural Network Transducer) (Graves, 2012) by an efficient forward-backward algorithm. However, RNN-T is trained based on the monotonic alignment between source and target sequences, which is not suitable for simultaneous translation, as it cannot properly consider reorder- ing. On the other hand, the forward-backward algo- rithm is not available for attention-based encoder- decoder (Bahdanau et al., 2015) architectures, in- cluding Transformer (Vaswani et al., 2017), due to the deep coupling between source contexts and target history contexts. To solve this problem, we separate the cross attention mechanism from target history represen- tation in attention-based encoder-decoder, which can also be viewed as RNN-T with the joiner being augmented by cross attention mechanism, resulting in Cross Attention Augmented Transducer (CAAT). However, cross attention mechanism removes the alignment constraint in RNN-T which originally encourages an appropriate latency. To ensure la- tency under control, jointly minimizing a latency …

Content truncated to 500 words