Building The First English-Brazilian Portuguese Corpus for Automatic Post-Editing

Proceedings of the 28th International Conference on Computational Linguistics, pages 6063–6069 Barcelona, Spain (Online), December 8-13, 2020 6063 Building The First English-Brazilian Portuguese Corpus for Automatic Post-Editing Felipe Almeida Costa, Thiago Castro Ferreira, Adriana Pagano, and Wagner Meira Jr Universidade Federal de Minas Gerais {felipealco,meira}@dcc.ufmg.br, {thiagocf05,apagano}@ufmg.br Abstract This paper introduces the first corpus for Automatic Post-Editing of English and a low-resource language, Brazilian Portuguese. The source English texts were extracted from the WebNLG corpus and automatically translated into Portuguese using a state-of-the-art industrial neural ma- chine translator. Post-edits were then obtained in an experiment with native speakers of Brazilian Portuguese. To assess the quality of the corpus, we performed error analysis and computed com- plexity indicators measuring how difficult the APE task would be. We report preliminary results of Phrase-Based and Neural Machine Translation Models on this new corpus. Data and code publicly available in our repository.1 1 Introduction Automatic post-editing (APE) is the computational task responsible for fixing systematic and repetitive errors found in black-box machine translation (MT) outputs (Vu and Haffari, 2018; Simard et al., 2007). APE is considered appealing for allowing the rapid and cheap customization of general-purpose machine translation models to specific application domains, avoiding the need for new systems to be trained from scratch (Correia and Martins, 2019). Moreover, these models deliver better machine-translated outputs to human translators, reducing human post-editing effort. APE systems are developed based on triples aligning source sentences, machine translation outputs and human post-edits. Data is gathered through interfaces where human translators can post-edit and improve the quality of machine-translated documents (Alabau et al., 2014; Federico et al., 2014). While human post-edits are necessary for training APE systems, there are not many publicly-available resources of this kind. For languages with pre-existing APE corpora, the scarcity of data has been solved by generating artificial triples (Junczys-Dowmunt and Grundkiewicz, 2016; Negri et al., 2018). However, even small APE corpora have not been publicly-available for most languages. For example, the WMT shared task on MT Automatic Post-Editing (APE), main source of studies in APE since 2015, provided an English-Spanish corpus in its 2015 version, an English-German APE corpus in its 2016-2018 versions (also a German-English in the 2017 one), and a novel English-Russian one in its 2019 version. In sum, only three languages have been explored in the main shared-task on APE. In order to fulfill the gap of data scarcity in different languages, we have compiled a novel corpus for automatic post-editing from English into Brazilian Portuguese, a low-resource language. Brazilian Portuguese has fewer resources available compared to European Portuguese, an official language of the European Union and, as such, featuring in several parallel corpora. The source English texts were extracted from the WebNLG corpus (Gardent et al., 2017), which also provides a meaning representation aligned to each text. The extracted texts were automatically translated into Portuguese using a state- of-the-art neural machine translation service. Native speakers of Brazilian Portuguese post-edited the machine-translated output through a web interface specifically designed for our experiment. …

Content truncated to 500 words