Xiaomi’s Submissions for IWSLT 2020 Open Domain Translation Task

Proceedings of the 17th International Conference on Spoken Language Translation (IWSLT), pages 149–157 July 9-10, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 149 Xiaomi’s Submissions for IWSLT 2020 Open Domain Translation Task Yuhui Sun, Mengxue Guo, Xiang Li, Jianwei Cui, Bin Wang AI Lab, Xiaomi Group {sunyuhui1,guomengxue1,lixiang21,cuijianwei,wangbin11}@xiaomi.com Abstract This paper describes the Xiaomi’s submissions to the IWSLT20 shared open domain transla- tion task for Chinese↔Japanese language pair. We explore different model ensembling strate- gies based on recent Transformer variants. We also further strengthen our systems via some first-line techniques, such as data filtering, data selection, tagged back translation, do- main adaptation, knowledge distillation, and re-ranking. Our resulting Chinese→Japanese primary system ranked second in terms of character-level BLEU score among all submis- sions. Our resulting Japanese→Chinese pri- mary system also achieved a competitive per- formance. 1 Introduction In this paper, we describe the Xiaomi’s neu- ral machine translation (NMT) systems eval- uated at IWSLT 2020 (Ansari et al., 2020) shared open domain translation task in two directions, Chinese→Japanese (Zh→Ja) and Japanese→Chinese (Ja→Zh). The accuracy of NMT systems relies on the qual- ity of training data, we first consider careful pre- processing and discard the corrupted data from the existing bilingual sentences according to rule-based filtering and model-based scoring. In the aspect of NMT architecture, we exploit some recent Transformer variants, including dif- ferent Transformer models with deeper layers or wider inner dimension of feed-forward layers than the standard Transformer-Big model, Trans- former with a dynamic linear combination of layers (DLCL) (Wang et al., 2019) and neural architecture search (NAS) based Transformer-Evolved (So et al., 2019), to increase the diversity of the system. We further strengthen our systems by diversifying the training data via some effective methods, includ- ing back-translation (BT) (Sennrich et al., 2016b), knowledge distillation (KD) (Hinton et al., 2015) and right-to-left (R2L) NMT model. Finally, we also explore re-rank the n-best translation candi- dates generated by models ensembling with some effective features, including target-to-source (T2S) NMT model, left-to-right (L2R) NMT model, R2L NMT model (Liu et al., 2016), bilingual sentence BERT and language model (LM). Through experiments, we evaluate how each sys- tem feature affects the accuracy of NMT. Our result- ing Chinese→Japanese primary system ranked sec- ond in terms of character-level BLEU score among all submissions. Our resulting Japanese→Chinese primary system also achieved a competitive perfor- mance. 2 Data 2.1 Pre-processing Our pre-processing pipeline begins by removing non-printable ASCII characters, lowercasing text, normalizing additional white-space, and control character and replacing any escaped characters with the corresponding symbol by our in-house script. All the data is further normalized so all full-width Roman characters and digits are normal- ized to half-width. All the traditional characters of Chinese data are converted to simplified characters using OpenCC1. For all corpora, Chinese sentences are segmented by our in-house Chinese word seg- menter, and Japanese sentences are first segmented by the morphological analyzer Mecab (Kudo, 2006) and then tokenized only for the non-Japanese part by the Moses script2. 1https://github.com/BYVoid/OpenCC 2https://github.com/moses-smt/ mosesdecoder/blob/master/scripts/ tokenizer/tokenizer.perl 150 2.2 …

Content truncated to 500 words