Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 1100–1109 November 10–11, 2021. ©2021 Association for Computational Linguistics 1100 Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora Gaurav Kumar, Philipp Koehn, Sanjeev Khudanpur Center for Language and Speech Processing Johns Hopkins University gkumar@cs.jhu.edu, {phi, khudanpur}@jhu.edu Abstract Large web-crawled corpora represent an excel- lent resource for improving the performance of Neural Machine Translation (NMT) sys- tems across several language pairs. However, since these corpora are typically extremely noisy, their use is fairly limited. Current ap- proaches to deal with this problem mainly fo- cus on filtering using heuristics or single fea- tures such as language model scores or bi- lingual similarity. This work presents an al- ternative approach which learns weights for multiple sentence-level features. These fea- ture weights which are optimized directly for the task of improving translation performance, are used to score and filter sentences in the noisy corpora more effectively. We provide results of applying this technique to building NMT systems using the Paracrawl corpus for Estonian-English and show that it beats strong single feature baselines and hand designed combinations. Additionally, we analyze the sensitivity of this method to different types of noise and explore if the learned weights gener- alize to other language pairs using the Maltese- English Paracrawl corpus. 1 Introduction Large parallel corpora such as Paracrawl (Bañón et al., 2020) which have been crawled from on- line resources hold the potential to drastically im- prove performance of neural machine translation systems across both low and high resource lan- guage pairs. However, since these extraction ef- forts mostly rely on automatic language identifica- tion and document/sentence alignment methods, the resulting corpora are extremely noisy. The most frequent noise types encountered are sentence alignment errors, wrong language in source or tar- get, and untranslated sentences. As outlined by Khayrallah and Koehn (2018), training algorithms for neural machine translation systems are partic- ularly vulnerable to these noise types. As such, these web-crawled corpora have seen limited use in training large NMT systems. This paper proposes a method for denoising and filtering noisy corpora which explores and searches over weighted combinations of features. During NMT training, we score sentences and cre- ate batches using random weight vectors. These batches are use to train the system and measure improvement over the validation set (reward). Fi- nally, by modeling the weight-reward function, we learn the set of weights which maximize reward and are used to score and filter the noisy dataset. At a high level, this method (i) allows the use of multiple sentence level features, (ii) learns a set of interpolation weights for the features which directly maximize translation performance, (iii) requires no prior knowledge about which features are informa- tive or even if they are mutually redundant, and (iv) trains within the NMT pipeline and does not require any special infrastructure. We include experiments which apply this method to building NMT systems for the noisy Estonian-English Paracrawl dataset and show that it beats strong single feature …

Content truncated to 500 words