Evaluating Corpus Cleanup Methods in the WMT’22 News Translation Task
27563Authors
nan
Abstract
AbstractThis submission to the WMT22: General MT Task, consists of translations produced from a series of NMT models of the following two language pairs: german-to-english and german-to-french. All the models are trained using only the parallel training data specified by WMT22, and no monolingual training data was used. The models follow the transformer architecture employing 8 attention heads and 6 layers in both the encoder and decoder. It is also worth mentioning that, in order to limit the computational resources that we would use during the training process, we decided to train the majority of models by limiting the training to 21 epochs. Moreover, the translations submitted at WMT22 have been produced using the test data released by the WMT22.The aim of our experiments has been to evaluate methods for cleaning-up a parallel corpus to determine if this will lead to a translation model producing more accurate translations. For each language pair, the base NMT models has been trained from raw parallel training corpora, while the additional NMT models have been trained with corpora subjected to a special cleaning process with the following tools: Bifixer and Bicleaner. It should be mentioned that the Bicleaner repository doesn’t provide pre-trained classifiers for the above language pairs, consequently we trained probabilistic dictionaries in order to produce new models. The fundamental differences between these NMT models produced are mainly related to the quality and the quantity of the training data, while there are very few differences in the training parameters. To complete this work, we used the following three software packages: (i) MARIAN NMT (Version: v1.11.5), which was used for the training of the neural machine translation models and (ii) Bifixer and (iii) Bicleaner, which were used in order to correct and clean the parallel training data. Concerning the Bifixer and Bicleaner tools, we followed all the steps as described meticulously in the following article: “Ramírez-Sánchez, G., Zaragoza-Bernabeu, J., Bañón, M., & Rojas, S.O. (2020). Bifixer and Bicleaner: two open-source tools to clean your parallel data. EAMT. ” and also in the official github pages: https://github.com/bitextor/bifixer, https://github.com/bitextor/bicleaner.
Keywords
Full Content
Proceedings of the Seventh Conference on Machine Translation (WMT), pages 335 - 341 December 7-8, 2022 ©2022 Association for Computational Linguistics Evaluating Corpus Cleanup Methods in the WMT’22 General Translation Task Marilena Malli Department of Informatics and Telecommunications, University of Athens Ilissia, 15784, Greece mallimariaeleni@gmail.com George Tambouratzis ILSP, Athena R.C. 6 Artemidos Str., Maroussi, 15125, Greece giorg_t@athenarc.gr Abstract This paper describes the LT’22 team’s con- strained submission to the WMT General Ma- chine Translation task. NMT transformer- based systems have been implemented using only the WMT’22 released parallel corpora, without using any pre-trained models. Two language pairs have been tackled, namely Ger- man to English and German to French. Empha- sis was placed on removing the noisy sections of parallel corpora where the degree of paral- lelism is very limited, for which a publicly- available tool was-used. Comparative results are reported with baseline systems. 1 Introduction This submission presents the contribution of the LT’22 team to the WMT22: General MT Task. It focuses on studying the effectiveness of cleaning tools when these are applied to real-world parallel corpora, to eliminate noisy sections and improve the resulting NMT systems. Traditionally, parallel corpora are used as the primary data source for machine translation (MT) models. The development of MT has been aided by the availability of extensive parallel corpora. The majority of these data have several areas of reduced parallelism and are usually characterized as imper- fect or noisy. The use of noisy data may result in a neural machine translation model being inade- quately prepared. Researchers (e.g. (Koehn and Knowles, 2017) (Khayrallah and Koehn, 2018)) have reported that neural machine translation mod- els are much more affected by noisy data than sta- tistical machine translation models. A number of software packages to implement noise-removal from parallel corpora have been im- plemented and released to the community. These include publicly available tools such as qe-clean (Denkowski)1, as well as Zipporah (Xu and Koehn, 1https://github.com/mjdenkowski/qe-clean 2017). (Zarin, a et al., 2015) have used a combi- nation of alignment-indicating features to clean corpora. For cleaning large-scale corpora in multi- lingual setups, a cosine-distance metric has been proposed (Schwenk and Li, 2018). Finally, the suite of the paired Bifixer and Bicleaner software tools (Ramírez-Sánchez et al., 2020) has been pro- posed for parallel corpora cleaning purposes, with Bifixer implementing restorative cleaning and Bi- cleaner providing the ability to remove sentences with very low parallelism in the parallel corpus. For the experiments reported here, two language pairs have been chosen, namely German-to-English (denoted as De-to-En) and German-to-French (de- noted as De-to-Fr). Compared with other systems reported in WMT, our NMTs have a couple of iden- tifying features: (1) the use of a fully-constrained setup with respect to WMT’22 rules and (2) the setting of a relatively low threshold to the allowed training epochs, in an effort to comply to a setup with limited computational resources. Whilst our translation systems are not as accurate as they could be if more epochs were allowed, it was decided to …
Content truncated to 500 words
Labeled Text
AbstractThis submission to the WMT22: General MT Task, consists of translations produced from a series of NMT models of the following two language pairs: german-to-english and german-to-french. All the models are trained using only the parallel training data specified by WMT22, and no monolingual training data was used. The models follow the transformer architecture employing 8 attention heads and 6 layers in both the encoder and decoder. It is also worth mentioning that, in order to limit the computational resources that we would use during the training process, we decided to train the majority of models by limiting the training to 21 epochs. Moreover, the translations submitted at WMT22 have been produced using the test data released by the WMT22.The aim of our experiments has been to evaluate methods for cleaning-up a parallel corpus to determine if this will lead to a translation model producing more accurate translations. For each language pair, the base NMT models has been trained from raw parallel training corpora, while the additional NMT models have been trained with corpora subjected to a special cleaning process with the following tools: Bifixer and Bicleaner. It should be mentioned that the Bicleaner repository doesn’t provide pre-trained classifiers for the above language pairs, consequently we trained probabilistic dictionaries in order to produce new models. The fundamental differences between these NMT models produced are mainly related to the quality and the quantity of the training data, while there are very few differences in the training parameters. To complete this work, we used the following three software packages: (i) MARIAN NMT (Version: v1.11.5), which was used for the training of the neural machine translation models and (ii) Bifixer and (iii) Bicleaner, which were used in order to correct and clean the parallel training data. Concerning the Bifixer and Bicleaner tools, we followed all the steps as described meticulously in the following article: “Ramírez-Sánchez, G., Zaragoza-Bernabeu, J., Bañón, M., & Rojas, S.O. (2020). Bifixer and Bicleaner: two open-source tools to clean your parallel data. EAMT. ” and also in the official github pages:
Publication Details
- Paper ID:
- 27563
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan