Empirical Evaluation of Language Agnostic Filtering of Parallel Data for Low Resource Languages

Empirical Evaluation of Language Agnostic Filtering of Parallel Data for Low Resource Languages Praveen Dakwale1 Talaat Khalil2 Brandon Denis3 Huawei Technologies R&D Amsterdam, Netherlands praveen.dakwale@huawei.com1, khalil.talaat@gmail.com2, brandon.james.denis@huawei.com3 Abstract Most of the available resources for low resource languages are crawled from the web. In order to obtain reasonable ma- chine translation performance with such datasets, it is important to filter low qual- ity samples from the training data. In this paper we explore the use of lan- guage agnostic sentence representations for filtering parallel data for low resource language pairs: Pashto-English, Khmer- English, Nepali-English and Sinhalese- English. We determine the quality of the samples based on embedding similarity be- tween source and target sentences. Our experiments show that when preceded by language filtering using language agnos- tic embeddings significantly improves the performance of neural machine translation (NMT) and achieve performance competi- tive to language specific approaches. 1 Introduction Neural machine Translation models are known to be data hungry (Koehn and Knowles, 2017). Train- ing a high-quality NMT model requires a very large amount of data, usually in the order of mil- lions of sentence pairs. For the majority of lan- guage pairs, parallel training data is compiled by aligning web-crawls in source and target languages using various heuristics based methods (Munteanu and Marcu, 2005). These web crawled datasets are often noisy due to alignment errors between source and target sentences. These misalignments © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. lead to models with poor translation performance (Khayrallah and Koehn, 2018). Therefore, it is im- portant to ensure the quality of the training data by filtering out noisy samples. Various filtering techniques have been proposed in the machine translation literature which focus on different types of noise. Significant gains can be achieved by applying simple rules that do not take into consideration the semantic similarity be- tween source and target sentences. Such rules in- clude: removal of sentence pairs that are identi- fied with language codes that are not aligned with source and target languages, length based pruning, removal of repetitive strings, and other heuristics (Barbu and Barbu Mititelu, 2018). However, a ma- jor type of noise is the non-equivalence of source and target sentences. Here, non-equivalence im- plies that the source and target sentences are not correct translations of each other (Khayrallah and Koehn, 2018). These non-equivalence cases are much harder to identify by simple heuristics. To detect noisy samples of this type, an oracle model is required to calculate the semantic similarity be- tween source and target sentence pairs. In the last few years, there has been a growing interest in developing advanced parallel data filter- ing techniques, resulting in a shared task for “par- allel corpus filtering” focusing on high and low re- source language pairs (Koehn et al., 2019; Koehn et al., 2020). The majority of these approaches fo- cus on rule-based pre-filtering followed by scoring with an oracle model trained on high-quality paral- lel …

Content truncated to 500 words