Corpora for Document-Level Neural Machine Translation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3775–3781 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3775 Corpora for Document-Level Neural Machine Translation Siyou Liu∗ Xiaojun Zhang† ∗ Macao Polytechnic Institute † Xi’an Jiaotong-Liverpool University violetal@ipm.edu.mo xiaojun.zhang01@xjtlu.edu.cn Abstract Instead of translating sentences in isolation, document-level machine translation aims to capture discourse dependencies across sentences by considering a document as a whole. In recent years, there have been more interests in modelling larger context for the state-of-the-art neural machine translation (NMT). Although various document-level NMT models have shown significant improvements, there nonetheless exist three main problems: 1) compared with sentence-level translation tasks, the data for training robust document-level models are relatively low-resourced; 2) experiments in previous work are conducted on their own datasets which vary in size, domain and language; 3) proposed approaches are implemented on distinct NMT architectures such as recurrent neural networks (RNNs) and self-attention networks (SANs). In this paper, we aim to alleviate the low-resource and under-universality problems for document-level NMT. First, we collect a large number of existing document-level corpora, which covers 7 language pairs and 6 domains. In order to address resource sparsity, we construct a novel document parallel corpus in Chinese–Portuguese, which is a non-English-centred and low-resourced language pair. Besides, we implement and evaluate the commonly-cited document-level method on top of the advanced Transformer model with universal settings. Finally, we not only demonstrate the effectiveness and univer- sality of document-level NMT, but also release the preprocessed data, source code and trained models for comparison and reproducibility. Keywords: Neural Machine Translation, Document-Level Translation, Corpus, Discourse 1. Introduction Neural machine translation (NMT) has been rapidly devel- oped in recent years (Kalchbrenner and Blunsom, 2013; Sutskever et al., 2014; Bahdanau et al., 2015). Con- ventional NMT models still translate a text by consider- ing isolated sentences, which may harm translation qual- ity especially in terms of coherence, cohesion, and consis- tency (Webber, 2014). To address this problem, document- level NMT has attracted increasing attention from the com- munity (Wang et al., 2017a; Jean et al., 2017; Kuang et al., 2017b; Bawden et al., 2018; Maruf and Haffari, 2018; Tu et al., 2018; Zhang et al., 2018; Miculicich et al., 2018; Voita et al., 2018; Xiong et al., 2019). Researchers have in- vestigated a variety of approaches to model cross-sentence context for NMT and shown promising results in terms of BLEU score (Papineni et al., 2002) as well as human evaluation (L¨aubli et al., 2018). For instance, Wang et al. (2017a) tried early attempt proposing a hierarchical neu- ral encoder to summarize context in previous sentences and then integrate the historical representations into standard NMT with three effective strategies. Although such approaches have achieved significant im- provements, there nonetheless exist three main drawbacks that may restrict development of document-level NMT. First, parallel documents are too scarce to build robust document-level NMT models. Taking Chinese–English translation task for example, the WMT2019 sentence-level corpus contains 25 million sentence pairs while …

Content truncated to 500 words