Phrase Clustering for Smoothing TM Probabilities - or, How to Extract Paraphrases from Phrase Tables

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 608–616, Beijing, August 2010 Phrase Clustering for Smoothing TM Probabilities – or, How to Extract Paraphrases from Phrase Tables 1Roland Kuhn, 1Boxing Chen, 1George Foster and 2Evan Stratford 1National Research Council of Canada 2University of Waterloo 1First.Last@nrc.gc.ca; 2evan.stratford@gmail.com Abstract This paper describes how to cluster to- gether the phrases of a phrase-based sta- tistical machine translation (SMT) sys- tem, using information in the phrase table itself. The clustering is symmetric and recursive: it is applied both to source- language and target-language phrases, and the clustering in one language helps determine the clustering in the other. The phrase clusters have many possible uses. This paper looks at one of these uses: smoothing the conditional translation model (TM) probabilities employed by the SMT system. We incorporated phrase-cluster-derived probability esti- mates into a baseline loglinear feature combination that included relative fre- quency and lexically-weighted condition- al probability estimates. In Chinese- English (C-E) and French-English (F-E) learning curve experiments, we obtained a gain over the baseline in 29 of 30 tests, with a maximum gain of 0.55 BLEU points (though most gains were fairly small). The largest gains came with me- dium (200-400K sentence pairs) rather than with small (less than 100K sentence pairs) amounts of training data, contrary to what one would expect from the pa- raphrasing literature. We have only be- gun to explore the original smoothing approach described here. 1 Introduction and Related Work The source-language and target-language “phras- es” employed by many statistical machine trans- lation (SMT) systems are anomalous: they are arbitrary sequences of contiguous words ex- tracted by complex heuristics from a bilingual corpus, satisfying no formal linguistic criteria. Nevertheless, phrase-based systems perform bet- ter than word-based systems (Koehn 2010, pp. 127-129). In this paper, we look at what happens when we cluster together these anomalous but useful entities. Here, we apply phrase clustering to obtain bet- ter estimates for “backward” probability P(s|t) and “forward” probability P(t|s), where s is a source-language phrase, t is a target-language phrase, and phrase pair (s,t) was seen at least once in training data. The current work is thus related to work on smoothing P(s|t) and P(t|s) – see (Foster et al., 2006). The relative frequency estimates for P(s|t) and P(t|s) are t st st PRF #(,)/# (|) = and s st ts PRF #(,)/# (|) = where #(s,t) denotes the number of times phrase , pair (s,t) was observed, etc. These estimates are typically smoothed with “lexical” estimates found by breaking phrases s and t into words. We adopt a different idea, that of smoothing PRF(s|t) and PRF(t|s) with estimates obtained from phrases that have similar meanings to s and t. In our experiments, the two methods were com- bined, yielding an improvement over lexical smoothing alone – this indicates they provide complementary information. E.g., lexical esti- mates don’t work well for non-compositional phrases like “kick the bucket” - our method might cluster this phrase with “die” and “expire” …

Content truncated to 500 words