Lattice Transformer for Speech Translation

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6475–6484 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 6475 Lattice Transformer for Speech Translation Pei Zhang∗, Boxing Chen∗, Niyu Ge∗, Kai Fan∗† Alibaba Group Inc. {xiaoyi.zp,boxing.cbx,niyu.ge,k.fan}@alibaba-inc.com Abstract Recent advances in sequence modeling have highlighted the strengths of the transformer ar- chitecture, especially in achieving state-of-the- art machine translation results. However, de- pending on the up-stream systems, e.g., speech recognition, or word segmentation, the input to translation system can vary greatly. The goal of this work is to extend the attention mechanism of the transformer to naturally consume the lattice in addition to the tradi- tional sequential input. We first propose a general lattice transformer for speech transla- tion where the input is the output of the au- tomatic speech recognition (ASR) which con- tains multiple paths and posterior scores. To leverage the extra information from the lat- tice structure, we develop a novel control- lable lattice attention mechanism to obtain la- tent representations. On the LDC Spanish- English speech translation corpus, our exper- iments show that lattice transformer general- izes significantly better and outperforms both a transformer baseline and a lattice LSTM. Additionally, we validate our approach on the WMT 2017 Chinese-English translation task with lattice inputs from different BPE segmen- tations. In this task, we also observe the im- provements over strong baselines. 1 Introduction Transformer based encoder-decoder framework (Vaswani et al., 2017) for Neural Machine Trans- lation (NMT) has currently become the state-of- the-art in many translation tasks, significantly im- proving translation quality in text (Bojar et al., 2018; Fan et al., 2018) as well as in speech (Jan et al., 2018). Most NMT systems fall into the cat- egory of Sequence-to-Sequence (Seq2Seq) model (Sutskever et al., 2014), because both the input and ∗indicates equal contribution. †corresponding author. 0: <s> 1: iban 2: ivan 6: esquinas 5: así 4: esquinas 3: espinas 7: así 8:entonces 9: </s> 0.87 0.87 1 0.13 0.13 1 0.13 0.11 1 0.87 0.76 0.87 1 0.11 0.11 1 0.13 0.13 1 0.89 0.89 1 1 1 1 1 1 1 1 1 x0 x1 x2 x6 x5 x4 x3 x7 x8 x9 0: <s> 1: iban 2: esquinas 3: así 4:entonces 5: </s> x0 x1 x2 x3 x4 x5 Attention in Standard Transformer Encoder Attention in Lattice Transformer Encoder Figure 1: Illustration of our proposed attention mech- anism (best viewed in color). Our attention depends on the tokens of common paths and forward (blue) / marginal (grey) / backward (orange) probability scores. output consist of sequential tokens. Therefore, in most neural speech translation, such as that of (Bo- jar et al., 2018), the input to the translation sys- tem is usually the 1-best hypothesis from the ASR instead of the word lattice output with its corre- sponding probability scores. How to consume word lattice rather than se- quential input has been substantially researched in several natural language processing (NLP) tasks, such as language modeling …

Content truncated to 500 words