HW-TSC’s Participation in the IWSLT 2022 Isometric Spoken Language Translation

Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 361 - 368 May 26-27, 2022 c⃝2022 Association for Computational Linguistics HW-TSC’s Participation in the IWSLT 2022 Isometric Spoken Language Translation Zongyao Li, Jiaxin Guo, Daimeng Wei, Hengchao Shang, Minghan Wang, Ting Zhu, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Lizhi Lei, Hao Yang, Ying Qin Huawei Translation Service Center, Beijing, China {lizongyao,guojiaxin1,weidaimeng,shanghengchao,wangminghan, zhuting20,wuzhanglin2,yuzhengzhe,chenxiaoyu35, leilizhi,yanghao30,qinying}@huawei.com Abstract This paper presents our submissions to the IWSLT 2022 Isometric Spoken Language Translation task. We participate in all three lan- guage pairs (English-German, English-French, and English-Spanish) under the constrained set- ting, and submit an English-German result un- der the unconstrained setting. We use the stan- dard Transformer model as the baseline and ob- tain the best performance via one of its variants that shares the decoder input and output em- bedding. We perform detailed pre-processing and filtering on the provided bilingual data. Several strategies are used to train our models, such as Multilingual Translation, Back Trans- lation, Forward Translation, R-Drop, Average Checkpoint, and Ensemble. We experiment on three methods for biasing the output length: i) conditioning the output to a given target- source length-ratio class; ii) enriching the trans- former positional embedding with length in- formation and iii) length control decoding for non-autoregressive translation etc. Our sub- missions achieve 30.7, 41.6 and 36.7 BLEU respectively on the tst-COMMON test sets for English-German, English-French, English- Spanish tasks and 100% comply with the length requirements. 1 Introduction This paper introduces our submissions to the IWSLT 2022 Isometric Spoken Language Trans- lation task. To train our models, we perform mul- tiple data filtering strategies to enhance data qual- ity. In addition, we leverage Multilingual model (Johnson et al., 2017), Forward (Wu et al., 2019) and Back Translation (Edunov et al., 2018), and R-Drop (Wu et al., 2021) strategies to further en- hance training effects. We also adopt Length Token (Lakew et al., 2019), Length Encoding (Takase and Okazaki, 2019) and Non-Autoregressive Transla- tion (NAT) to further enhance system performances. We compare and contrast different strategies in Length-aware beam Enhanced model Length token Method Length encoding Method NAT Rerank Figure 1: The training process for the IWSLT 2022 Isometric Spoken Language Translation. light of our experiment results and conduct analy- sis accordingly. The overall training process is illustrated in Fig- ure 1. Section 2 focuses on our training techniques, including model architecture, data processing and training strategies. Section 3 describes our ex- periment settings and training process. Section 4 presents the experiment results while section 5 analyzes the effects of different model enhance- ment and length control strategies on the quality and length of translation outputs. 2 Method 2.1 Model Architecture 2.1.1 Autoregressive NMT Model Transformer-based model with the self-attention mechanism (Vaswani et al., 2017) has achieved the state-of-the-art translation performance. The Trans- former architecture is a standard encoder-decoder model. The encoder can be viewed as a stack of N layers, including a self-attention sub-layer and a feed-forward (FFN) sub-layer. The decoder shares a similar architecture …

Content truncated to 500 words