A Double-Graph Based Framework for Frame Semantic Parsing

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 494 - 507 July 10-15, 2022 ©2022 Association for Computational Linguistics Automatic Correction of Human Translations Jessy Lin♢♣ Geza Kovacs♡ Aditya Shastry♣ Joern Wuebker♣ John DeNero♢♣ ♢ University of California, Berkeley ♡ Google ♣ Lilt jessy_lin@berkeley.edu, joern@lilt.com, john@lilt.com Abstract We introduce translation error correction (TEC), the task of automatically correcting human-generated translations. Imperfections in machine translations (MT) have long moti- vated systems for improving translations post- hoc with automatic post-editing. In contrast, little attention has been devoted to the problem of automatically correcting human translations, despite the intuition that humans make distinct errors that machines would be well-suited to assist with, from typos to inconsistencies in translation conventions. To investigate this, we build and release the ACED corpus with three TEC datasets1. We show that human errors in TEC exhibit a more diverse range of errors and far fewer translation fluency errors than the MT errors in automatic post-editing datasets, suggesting the need for dedicated TEC mod- els that are specialized to correct human errors. We show that pre-training instead on synthetic errors based on human errors improves TEC F-score by as much as 5.1 points. We con- ducted a human-in-the-loop user study with nine professional translation editors and found that the assistance of our TEC system led them to produce significantly higher quality revised translations. 1 Introduction Despite recent progress in machine translation (MT), a tremendous amount of translated content in the world is still written by humans (DePalma, 2021). Humans are often assumed to produce trusted, high-quality translations. In reality, they do make errors, including spelling, grammar, and translation errors (Hansen, 2009). This paper in- troduces the task of translation error correction (TEC). Given a source sentence s and a human- generated translation t, the goal of TEC is to pro- duce an improved translation t′ by correcting all errors in t. 1Dataset available at: github.com/lilt/tec “Translation correction” has long been studied in the MT community through the task of auto- matic post-editing (APE), which aims to correct errors in machine-generated translations (Simard et al., 2007). TEC is structurally identical to APE. However, it requires modeling a different data dis- tribution: errors made by humans, which differ from those made by MT systems (Freitag et al., 2021). We characterize the error distribution in TEC by building, analyzing, and releasing the ACED corpus, a collection of three TEC datasets from varying domains, with a total of 35,261 En- glish–German translations produced and corrected by professional translators in the natural course of their work. While APE is dominated by the fluency errors that are characteristic of MT systems (74% of sentences), our TEC corpus exhibits a broader distribution of errors that human translators are prone to make. Using this error analysis, we propose an ap- proach for TEC that pre-trains on synthetic cor- ruptions more similar to errors made by humans, outperforming models that were developed for the related tasks …

Content truncated to 500 words