Correct Me If You Can: Learning from Error Corrections and Markings

Correct Me If You Can: Learning from Error Corrections and Markings Julia Kreutzer∗ and Nathaniel Berger∗ and Stefan Riezler†,∗ ∗Computational Linguistics & †IWR Heidelberg University, Germany {kreutzer, berger, riezler}@cl.uni-heidelberg.de Abstract Sequence-to-sequence learning involves a trade-off between signal strength and an- notation cost of training data. For ex- ample, machine translation data range from costly expert-generated translations that enable supervised learning, to weak quality-judgment feedback that facilitate reinforcement learning. We present the first user study on annotation cost and machine learnability for the less popu- lar annotation mode of error markings. We show that error markings for trans- lations of TED talks from English to German allow precise credit assignment while requiring significantly less human effort than correcting/post-editing, and that error-marked data can be used success- fully to fine-tune neural machine transla- tion models. 1 Introduction Successful machine learning for structured output prediction requires the effort of annotating suf- ficient amounts of gold-standard outputs—a task that can be costly if structures are complex and ex- pert knowledge is required, as for example in neu- ral machine translation (NMT) (Bahdanau et al., 2015). Approaches that propose to train sequence- to-sequence prediction models by reinforcement learning from task-specific scores, for example BLEU in machine translation (MT), shift the prob- lem by simulating such scores by evaluating ma- chine translation output against expert-generated c⃝ 2020 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. reference structures (Ranzato et al., 2016; Bah- danau et al., 2017; Kreutzer et al., 2017; Sokolov et al., 2017). An alternative approach that proposes to considerably reduce human annotation effort by allowing to mark errors in machine outputs, for ex- ample erroneous words or phrases in a machine translation, has recently been proposed and been investigated in simulation studies by Marie and Max (2015); Domingo et al. (2017); Petrushkov et al. (2018). This approach takes the middle ground between supervised learning from error corrections as in machine translation post-editing1 (or from translations created from scratch) and reinforcement learning from sequence-level ban- dit feedback (this includes self-supervised learning where all outputs are rewarded uniformly). Error markings are highly promising since they suggest an interaction mode with low annotation cost, yet they can enable precise token-level credit/blame assignment, and thus can lead to an effective fine- grained discriminative signal for machine learning and data filtering. Our work is the first to investigate learning from error markings in a user study. Error corrections and error markings are collected from junior pro- fessional translators, analyzed, and used as train- ing data for fine-tuning neural machine translation systems. The focus of our work is on the learn- ability from error corrections and error markings, and on the behavior of annotators as teachers to a machine translation system. We find that error markings require significantly less effort (in terms of key-stroke-mouse-ratio (KSMR) and time) and result in a lower correction rate (ratio of words marked as incorrect or corrected in a post-edit). Furthermore, they are less prone …

Content truncated to 500 words