APE-QUEST: an MT Quality Gate

APE-QUEST: a Quality Gate for Routing MT Joachim Van den Bogaert, Heidi Depraetere, Sara Szoc, Tom Vanallemeersch CrossLang, Kerkstraat 106, 9050 Gentbrugge, Belgium {first.lastname}@crosslang.com Lucia Specia, Julia Ive University of Sheffield {initial.lastname}@sheffield.ac.uk Maxim Khalilov Unbabel, Rua Visconde de Santarém, 67B, 1000-286 Lisboa, Portugal maxim@unbabel.com Abstract The APE-QUEST project (2018–2020) provides a quality gate connected to the eTranslation system of EC’s Connecting Europe Facility (CEF). The quality gate supports translation in specific domains and involves quality estimation (QE), au- tomatic post-edition (APE) of machine translation (MT) output, human post- editing (PE) and secure data transfer. Public PE datasets are provided. Evalua- tions involving three language pairs are ongoing. 1 Overview The APE-QUEST project (Automated Post- editing and Quality Estimation) is funded by the EC’s CEF Telecom programme (project 2017- EU-IA-0151) which started in October 2018 and runs until September 2020. The project offers a quality gate connected to the eTranslation MT system, developed by the Directorate-General for Translation and provided by the CEF Automated Translation building block of the Directorate- General for Communications Networks, Content and Technology (DG CNECT) as a service to Digital Service Infrastructures (DSIs) of the EC and to public administrations of Member States. The project consortium consists of two companies, CrossLang (coordinator) and Unbabel, and the University of Sheffield. © 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, at- tribution, CC-BY-ND. APE-QUEST consists of a quality gate which injects QE and APE into the translation work- flow. The main goals of this injection are (1) to reach the desired translation quality in an effi- cient and reliable way using MT and PE and (2) to create data aggregation opportunities by mak- ing translations and post-edits “locally owned”, as data is generated and curated at the end user’s site, in accordance with the similar main princi- ple of the EC’s ELRC action.1 The APE-QUEST project focuses on mature technologies by integrating systems for MT, QE and APE. Tests involve three language pairs (English to Portuguese, French and Dutch) and three domains (legal-domain text, procurement and online dispute resolution). 2 Architecture The workflow consists of three tiers: (1) MT output with acceptable quality flows directly to the end user or connected system, (2) moderate- quality MT is enhanced through APE, and (3) low-quality MT is sent to a workflow for human PE. The input consists of text snippets (messages) or full text documents from the project’s stakeholders, such as DSIs, public services in Member States, and organisations involved in CEF Telecom projects adopting eTranslation. The input is provided through an API or user interface and is segmented into sentences to allow for routing sentences to different tiers based on the detected quality. 1 http://lr-coordination.eu (European Language Resources Coordination) Figure 1 Architecture of APE-QUEST The PE workflow allows for collecting user data for system improvement, i.e. data for adapt- ing the MT system and re-training QE and APE systems. The components of the workflow have been successfully integrated and it is fully …

Content truncated to 500 words