Data Selection With Fewer Words

Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 58–65, Lisboa, Portugal, 17-18 September 2015. c⃝2015 Association for Computational Linguistics. Data Selection With Fewer Words Amittai Axelrod University of Maryland amittai@umd.edu Xiaodong He Microsoft Research xiaohe@microsoft.com Philip Resnik University of Maryland resnik@umd.edu Mari Ostendorf University of Washington ostendo@uw.edu Abstract We present a method that improves data selection by combining a hybrid word/part-of-speech representation for corpora, with the idea of distinguishing between rare and frequent events. We validate our approach using data selection for machine translation, and show that it maintains or improves BLEU and TER translation scores while substantially im- proving vocabulary coverage and reducing data selection model size. Paradoxically, the coverage improvement is achieved by abstracting away over 97% of the total training corpus vocabulary using simple part-of-speech tags during the data selection process. 1 Introduction Data selection uses a small set of domain-relevant data to select additional training items from a much larger, out-of-domain dataset. Its goal is to filter Big Data down to Good Data: finding the best, most relevant data to use to train a model for a particular task. The prevalent data selection method, cross- entropy difference (Moore and Lewis, 2010), can produce domain-specific systems that are usually as good as or better than systems using all avail- able training data (Axelrod et al., 2011). The size of these domain-specific systems scales roughly linearly with the amount of selected data: a system trained on the most domain-relevant 10% of the full out-of-domain dataset will be only one tenth of the size of a system trained using all the avail- able data. This can be a large win in settings where training time matters, and also where compactness of the final system matters, e.g. running speech recognition or translation on mobile devices. While data selection thus eliminates the need to train systems on the entire pool of available data, the data selection process itself does not scale well (it still requires a language model built on the en- tire pool) and, more significantly, it comes at a cost: training on selected subsets leads to reduc- tions in vocabulary coverage compared to training on the full out-of-domain data pool. This coverage is important, because most NLP systems face the problem of handling words that were not seen in training the system, i.e. out-of-vocabulary (OOV) words. In automatic speech recognition (ASR), for example, OOV words pose a substantial problem, since the system will hallucinate a phonetically similar word in its vocabulary when an OOV word is encountered. In machine translation (MT), our focal application in this paper, OOVs can some- times be transliterated, but often they are ignored or passed through without translation, and gaps in vocabulary coverage can have a significant effect on MT performance (Daum´e III and Jagarlamudi, 2011; Irvine and Callison-burch, 2013). We introduce a method that preserves the data selection benefit of reducing translation system size. Our method performs as well or better than the standard cross-entropy difference method, as measured by downstream …

Content truncated to 500 words