The University of Maryland’s Submissions to the WMT20 Chat Translation Task: Searching for More Data to Adapt Discourse-Aware Neural Machine Translation

Proceedings of the 5th Conference on Machine Translation (WMT), pages 456–461 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 456 The University of Maryland’s Submissions to the WMT20 Chat Translation Task: Searching for More Data to Adapt Discourse-Aware Neural Machine Translation Calvin Bao∗ Yow-Ting Shiue∗ Chujun Song Jie S. Li Marine Carpuat Department of Computer Science, University of Maryland, College Park, MD {csbao,ytshiue,cjsong,jli2718,marine}@cs.umd.edu Abstract This paper describes the University of Mary- land’s submissions to the WMT20 Shared Task on Chat Translation. We focus on translat- ing agent-side utterances from English to Ger- man. We started from an off-the-shelf BPE- based standard transformer model trained with WMT17 news and fine-tuned it with the pro- vided in-domain training data. In addition, we augment the training set with its best matches in the WMT19 news dataset. Our primary submission uses a standard Transformer, while our contrastive submissions use multi-encoder Transformers to attend to previous utterances. Our primary submission achieves 56.7 BLEU on the agent side (en→de), outperforming a baseline system provided by the task organiz- ers by more than 13 BLEU points. Moreover, according to an evaluation on a set of carefully- designed examples, the multi-encoder archi- tecture is able to generate more coherent trans- lations. 1 Introduction Recent advances have made MT a widespread tool for asynchronous consumption of text. The dream of dissolving language barriers, however, will not be fulfilled until MT enables two or more people carry on a synchronous conversation, each speak- ing their native languages. Building translation systems that enable seamless conversations be- tween an English-speaking customer support agent and a German-speaking customer is the goal of WMT20’s shared task of chat translation (Farajian et al., 2020). In participation of this shared task, we focused on the agent side, translating English utterances into German. Our methods are inspired by Voita et al. (2018) and Bawden et al. (2018), ex- plicitly leveraging broader context to address coref- erence and cohesion to improve translation quality. ∗Equal contribution. We compare architectures of a standard transformer with a single encoder and a multi-encoder one with an additional transformer encoder to incorporate information from the previous utterance. In the case of blind testing or production use, since cus- tomer target utterances (English) will not be given, a separate de→en model was trained and used to back-translate customer utterances. Additionally, given the limited training pairs, we experiment with augmenting our dataset. We se- lected a subset of WMT19 en-de news data that were similar to the chat training data, which we then added to the training data. The subset was constructed using a full-text search engine loaded with the entire en-de WMT19 news data, which iterated through each chat training example, query- ing for the two closest matches with both the source and target as search strings. Our primary system, denoted PRIMARY, is a single-encoder pretrained transformer fine-tuned on WMT20 Chat data. The first contrastive sys- tem, denoted CONTRASTIVE1, is a multi-encoder transformer that pre-warms, using WMT19 news data, the weights of an …

Content truncated to 500 words