A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing

Proceedings of the First Conference on Machine Translation, Volume 1: Research Papers, pages 110–117, Berlin, Germany, August 11-12, 2016. c⃝2016 Association for Computational Linguistics A Comparative Study on Vocabulary Reduction for Phrase Table Smoothing Yunsu Kim, Andreas Guta, Joern Wuebker∗, and Hermann Ney Human Language Technology and Pattern Recognition Group RWTH Aachen University, Aachen, Germany {surname}@cs.rwth-aachen.de ∗Lilt, Inc. joern@lilt.com Abstract This work systematically analyzes the smoothing effect of vocabulary reduction for phrase translation models. We ex- tensively compare various word-level vo- cabularies to show that the performance of smoothing is not significantly affected by the choice of vocabulary. This result provides empirical evidence that the stan- dard phrase translation model is extremely sparse. Our experiments also reveal that vocabulary reduction is more effective for smoothing large-scale phrase tables. 1 Introduction Phrase-based systems for statistical machine trans- lation (SMT) (Zens et al., 2002; Koehn et al., 2003) have shown state-of-the-art performance over the last decade. However, due to the huge size of phrase vocabulary, it is difficult to collect robust statistics for lots of phrase pairs. The standard phrase translation model thus tends to be sparse (Koehn, 2010). A fundamental solution to a sparsity problem in natural language processing is to reduce the vo- cabulary size. By mapping words onto a smaller label space, the models can be trained to have denser distributions (Brown et al., 1992; Miller et al., 2004; Koo et al., 2008). Examples of such la- bels are part-of-speech (POS) tags or lemmas. In this work, we investigate the vocabulary re- duction for phrase translation models with respect to various vocabulary choice. We evaluate two types of smoothing models for phrase translation probability using different kinds of word-level la- bels. In particular, we use automatically gener- ated word classes (Brown et al., 1992) to obtain label vocabularies with arbitrary sizes and struc- tures. Our experiments reveal that the vocabulary of the smoothing model has no significant effect on the end-to-end translation quality. For exam- ple, a randomized label space also leads to a de- cent improvement of BLEU or TER scores by the presented smoothing models. We also test vocabulary reduction in transla- tion scenarios of different scales, showing that the smoothing works better with more parallel cor- pora. 2 Related Work Koehn and Hoang (2007) propose integrating a la- bel vocabulary as a factor into the phrase-based SMT pipeline, which consists of the following three steps: mapping from words to labels, label- to-label translation, and generation of words from labels. Rishøj and Søgaard (2011) verify the ef- fectiveness of word classes as factors. Assuming probabilistic mappings between words and labels, the factorization implies a combinatorial expan- sion of the phrase table with regard to different vocabularies. Wuebker et al. (2013) show a simplified case of the factored translation by adopting hard assign- ment from words to labels. In the end, they train the existing translation, language, and reordering models on word classes to build the corresponding smoothing models. Other types of features are also trained on word- …

Content truncated to 500 words