An Efficient Method for Determining Bilingual Word Classes

Proceedings of EACL '99 An Efficient Method for Determining Bilingual Word Classes Franz Josef Och Lehrstuhl ffir Informatik VI RWTH Aachen - University of Technology Ahornstrai3e 55 52056 Aachen GERMANY och@informatik.rwth-aachen.de Abstract In statistical natural language process- ing we always face the problem of sparse data. One way to reduce this problem is to group words into equivalence classes which is a standard method in statistical language modeling. In this paper we de- scribe a method to determine bilingual word classes suitable for statistical ma- chine translation. We develop an opti- mization criterion based on a maximum- likelihood approach and describe a clus- tering algorithm. We will show that the usage of the bilingual word classes we get can improve statistical machine transla- tion. 1 Introduction Word classes are often used in language modelling to solve the problem of sparse data. Various clus- tering techniques have been proposed (Brown et al., 1992; Jardino and Adda, 1993; Martin et al., 1998) which perform automatic word clustering optimizing a maximum-likelihood criterion with iterative clustering algorithms. In the field of statistical machine translation we also face the problem of sparse data. Our aim is to use word classes in statistical machine translation to allow for more robust statistical translation models. A naive approach for doing this would be the use of mono-lingually optimized word classes in source and target language. Un- fortunately we can not expect these independently optimized classes to be correspondent. There- fore mono-lingually optimized word classes do not seem to be useful for machine translation (see also (Fhng and Wu, 1995)). We define bilingual word clustering as the process of forming correspond- ing word classes suitable for machine translation purposes for a pair of languages using a parallel training corpus. The described method to determine bilingual word classes is an extension and improvement of the method mentioned in (Och and Weber, 1998). Our approach is simpler and computation- ally more efficient than (Wang et al., 1996). 2 Monolingual Word Clustering The task of a statistical language model is to es- timate the probability Pr(w N) of a sequence of words w N = wl...wg. A simple approximation of Pr(w N) is to model it as a product of bigram probabilities: Pr(w~) N = I-[i=1P(WilWi-1) • If we want to estimate the bigram probabilities p(w[w') using a realistic natural language corpus we are faced with the problem that most of the bigrams are rarely seen. One possibility to solve this prob- lem is to partition the set of all words into equiv- alence classes. The function C maps words w to their classes C(w). Rewriting the corpus probabil- ity using classes we arrive at the following proba- bility model p(wNlC): N P(wNIC) := i=1 (1) In this model we have two types of probabili- ties: the transition probability p(CIC ~) for class C given its predecessor class C' and the member- ship probability p(wlC ) for word w given class C. To determine the optimal classes C for a given number …

Content truncated to 500 words