Document Sub-structure in Neural Machine Translation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3657–3667 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3657 Document Sub-structure in Neural Machine Translation Radina Dobreva1, Jie Zhou2, Rachel Bawden1 1School of Informatics, University of Edinburgh, 10 Crichton Street, Edinburgh, United Kingdom 2Alibaba Group, 969 West Wen Yi Road, Hangzhou, China rdobreva@ed.ac.uk, zj236040@alibaba-inc.com, rbawden@ed.ac.uk Abstract Current approaches to machine translation (MT) either translate sentences in isolation, disregarding the context they appear in, or model context at the level of the full document, without a notion of any internal structure the document may have. In this work we consider the fact that documents are rarely homogeneous blocks of text, but rather consist of parts covering different topics. Some documents, such as biographies and encyclopedia entries, have highly predictable, regular structures in which sections are characterised by different topics. We draw inspiration from Louis and Webber (2014) who use this information to improve statistical MT and transfer their proposal into the framework of neural MT. We compare two different methods of including information about the topic of the section within which each sentence is found: one using side constraints and the other using a cache-based model. We create and release the data on which we run our experiments – parallel corpora for three language pairs (Chinese-English, French-English, Bulgarian-English) from Wikipedia biographies, which we extract automatically, preserving the boundaries of sections within the articles. Keywords: machine translation, document structure, corpus creation, context, Wikipedia, parallel corpus 1 Introduction Considerable progress has been made in machine transla- tion (MT) thanks to the use of neural MT (NMT). While most NMT systems translate at the level of individual sen- tences, following similar practices in statistical MT (SMT), there has been significant interest in recent years in using document context to improve translation (Hardmeier, 2014; Bawden, 2018; Wang, 2019). However the intermediate level of the internal structure of documents, particularly for documents with regular sub-structure, could also provide useful information to improve MT. Documents are rarely without internal structure, and certain document types (e.g. biographies, scientific articles and en- cyclopedia entries) are characterised by a more well-defined and regular structure than others. In such documents, the structure is explicitly defined by sections associated with headings dealing with different aspects of the main subject. These sections, many of which can be found across multi- ple documents, are often associated with specific vocabu- laries or even grammatical patterns. Figure 1 illustrates the classic structure of a Wikipedia article, with section head- ings (e.g. “early life”, “career”, “personal life”, etc.) that are likely to be found in other articles concerning high-profile people. This regularity in document sub-structure could be beneficial to the MT of such documents by providing ad- ditional information about the type of vocabulary used in different sections. Exploiting document sub-structure in MT has previously been studied by Louis and Webber (2014) for SMT. They use topic models to integrate section information into a cache-based system and see …

Content truncated to 500 words