Bicleaner AI: Bicleaner Goes Neural

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 824–831 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 824 Bicleaner AI: Bicleaner Goes Neural Jaume Zaragoza-Bernabeu, Marta Ba˜n´on, Gema Ram´ırez-S´anchez, Sergio Ortiz-Rojas Prompsit Language Engineering, SL (PLE), Spain {jzaragoza, mbanon, gramirez, sortiz}@prompsit.com Abstract This paper describes the experiments carried out during the development of the latest version of Bicleaner, named Bicleaner AI, a tool that aims at detecting noisy sentences in parallel corpora. The tool, which now implements a new neural classifier, uses state-of-the-art techniques based on pre-trained transformer-based language models fine-tuned on a binary classification task. After that, parallel corpus filtering is performed, discarding the sentences that have lower probability of being mutual translations. Our experiments, based on the training of neural machine translation (NMT) with corpora filtered using Bicleaner AI for two different scenarios, show significant improvements in translation quality compared to the previous version of the tool which implemented a classifier based on Extremely Randomized Trees. Keywords: Machine Translation, Corpus, Tools, Evaluation Methodologies 1. Introduction Parallel corpus filtering has become an important sub- task of machine translation, specially on web-crawled corpora, due to the noisy nature of raw crawls. In the past few years, the WMT Parallel Corpus Filtering Shared Task (Koehn et al., 2018; Koehn et al., 2019; Koehn et al., 2020) has shown the importance of data cleaning for neural machine translation (NMT). Previ- ous studies were warning us about this specific need, in contrast to what happens with statistical machine trans- lation (Khayrallah and Koehn, 2018). Bicleaner1 is a tool that aims at detecting noisy sen- tences in parallel corpora, indicating, for each sentence pair, the likelihood of being mutual translations. In previous submissions (S´anchez-Cartagena et al., 2018; Espl`a-Gomis et al., 2020), Bicleaner achieved compa- rable results to the state-of-the-art. Also, it has been the default cleaning step in the ParaCrawl (Ba˜n´on et al., 2020) pipeline since the third release of the corpus. The ParaCrawl pipeline is based on Bitextor, 2 a tool to automatically harvest bitexts from multilingual web- sites including Bicleaner as a component. Despite dis- carding a considerable amount of noisy sentences from ParaCrawl and improving machine translation quality, manual observation of the corpus still indicated that further cleaning was needed to bring out the full po- tential from the data. In order to take a step forward and catch up with the state of the art, we have developed a new version of the tool that significantly improves its accuracy compared to previous versions. Following the steps of Ac¸arc¸ic¸ek et al. (2020), who approached the problem similarly to how Bicleaner does (a supervised task where already existing parallel sentences are positive samples, and synthetically generated misaligned sentences are neg- ative samples), we replaced a classifier based on hand- crafted features and extremely randomized trees with a 1https://github.com/bitextor/bicleaner 2https://github.com/bitextor/bitextor fine-tuned XLM-RoBERTa classifier. The new version has been called Bicleaner AI and it is publicly available in a separated repository. 3 To assess …

Content truncated to 500 words