Data Selection with Cluster-Based Language Difference Models and Cynical Selection

Data Selection with Cluster-Based Language Difference Models and Cynical Selection Luc´ıa Santamar´ıa, Amittai Axelrod Amazon.com {lucsan,amittai}@amazon.com Abstract We present and apply two methods for addressing the problem of selecting relevant training data out of a general pool for use in tasks such as machine translation. Build- ing on existing work on class-based language difference models [1], we first introduce a cluster-based method that uses Brown clusters to condense the vocabulary of the cor- pora. Secondly, we implement the cynical data selection method [2], which incrementally constructs a training corpus to efficiently model the task corpus. Both the cluster-based and the cynical data selection approaches are used for the first time within a machine translation system, and we perform a head-to-head comparison. Our intrinsic evaluations show that both new meth- ods outperform the standard Moore-Lewis approach (cross- entropy difference), in terms of better perplexity and OOV rates on in-domain data. The cynical approach converges much quicker, covering nearly all of the in-domain vocab- ulary with 84% less data than the other methods. Furthermore, the new approaches can be used to se- lect machine translation training data for training better sys- tems. Our results confirm that class-based selection using Brown clusters is a viable alternative to POS-based class- based methods, and removes the reliance on a part-of-speech tagger. Additionally, we are able to validate the recently pro- posed cynical data selection method, showing that its per- formance in SMT models surpasses that of traditional cross- entropy difference methods and more closely matches the sentence length of the task corpus. 1. Data Selection, Previously 1.1. Moore-Lewis Data Selection The standard data selection method of Moore and Lewis [3] uses cross-entropy difference as the similarity metric to esti- mate the relevance of each sentence in the general pool cor- pus. This method takes advantage of the presumed mismatch between the pool data and the task domain. It first trains an in-domain language model (LM) on the task data, and then trains another LM on the full pool of general data. The aver- age per-word perplexity of each sentence in the pool data is computed relative to each of these models. The cross-entropy Hlm(s) of a sentence s, according to language model lm, is the log of the perplexity of the language model on that sen- tence. The cross-entropy difference score of [3] is: HLMT ASK(s) − HLMP OOL(s). Sentences that are most like the task data, and most un- like an average sentence in the full pool will have lower cross-entropy difference scores. A modification of this method, the bilingual Moore-Lewis criterion [4] used for se- lecting bilingual data for machine translation. This is a sim- ple extension, combining the cross-entropy difference scores from each side of the corpus; i.e. for sentence pair hs1, s2i � HLMT ASK1 (s1) − HLMP OOL1 (s1) � + � HLMT ASK2 (s2) − HLMP OOL2 (s2) � . For both the regular and bilingual Moore-Lewis methods, data selection is performed by sorting the sentences accord- ing to …

Content truncated to 500 words