Does more data always yield better translations?

Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 152–161, Avignon, France, April 23 - 27 2012. c⃝2012 Association for Computational Linguistics Does more data always yield better translations? Guillem Gasc´o, Martha-Alicia Rocha, Germ´an Sanchis-Trilles, Jes´us Andr´es-Ferrer and Francisco Casacuberta Departament de Sistemes Inform`atics i Computaci´o Universitat Polit`ecnica de Val`encia Cam´ı de Vera s/n, 46022 Val`encia, Spain {ggasco,mrocha,gsanchis,jandres,fcn}@dsic.upv.es Abstract Nowadays, there are large amounts of data available to train statistical machine trans- lation systems. However, it is not clear whether all the training data actually help or not. A system trained on a subset of such huge bilingual corpora might outperform the use of all the bilingual data. This paper studies such issues by analysing two train- ing data selection techniques: one based on approximating the probability of an in- domain corpus; and another based on in- frequent n-gram occurrence. Experimental results not only report significant improve- ments over random sentence selection but also an improvement over a system trained with the whole available data. Surprisingly, the improvements are obtained with just a small fraction of the data that accounts for less than 0.5% of the sentences. After- wards, we show that a much larger room for improvement exists, although this is done under non-realistic conditions. 1 Introduction Globalisation and the popularisation of the Inter- net have lead to a rapid increase in the amount of bilingual corpora available. Entities such as the European Union, the United Nations and other multinational organisations need to translate all the documentation they generate. Such transla- tions happen every day and provide very large multilingual corpora, which are oftentimes diffi- cult to process and significantly increase the com- putational requirements needed to train statistical machine translation (SMT) systems. For instance, the corpora made available for recent machine translation evaluations are in the order of 1 billion running words (Callison-Burch et al., 2010). However, two main problems arise when at- tempting to use this huge pool of sentences for training SMT systems: firstly, a large portion of this data is obtained from domains that differ from that in which the SMT system is to be used or as- sessed; secondly, the use of all this data for train- ing the system increases the computational train- ing requirements. Despite the previous remarks, the de facto standard consists in training SMT sys- tems with all the available data. This is due to the widespread misconception that the more data a system is trained with, the better its performance should be. Although the previous statement is the- oretically true if all the data belongs to the same domain, this is not the case in the problems tack- led by most of the SMT systems. For instance, enterprises often need to build on-demand sys- tems (Yuste et al., 2010). In this case, since we are interested in translating some specific text, it is not clear whether training a system with all data yields better performance than training it with a wisely selected …

Content truncated to 500 words