Base Noun Phrase Translation Using Web Data and the EM Algorithm

Base Noun Phrase Translation Using Web Data and the EM Algorithm Yunbo Cao Microsoft Research Asia i-yuncao@microsoft.com Hang Li Microsoft Research Asia hangli@microsoft.com Abstract We consider here the problem of Base Noun Phrase translation. We propose a new method to perform the task. For a given Base NP, we first search its translation candidates from the web. We next determine the possible translation(s) from among the candidates using one of the two methods that we have developed. In one method, we employ an ensemble of Naïve Bayesian Classifiers constructed with the EM Algorithm. In the other method, we use TF-IDF vectors also constructed with the EM Algorithm. Experimental results indicate that the coverage and accuracy of our method are significantly better than those of the baseline methods relying on existing technologies. 1. Introduction We address here the problem of Base NP translation, in which for a given Base Noun Phrase in a source language (e.g., ‘information age’ in English), we are to find out its possible translation(s) in a target language (e.g., ‘��� �’ in Chinese). We define a Base NP as a simple and non-recursive noun phrase. In many cases, Base NPs represent holistic and non-divisible concepts, and thus accurate translation of them from one language to another is extremely important in applications like machine translation, cross language information retrieval, and foreign language writing assistance. In this paper, we propose a new method for Base NP translation, which contains two steps: (1) translation candidate collection, and (2) translation selection. In translation candidate collection, for a given Base NP in the source language, we look for its translation candidates in the target language. To do so, we use a word-to-word translation dictionary and corpus data in the target language on the web. In translation selection, we determine the possible translation(s) from among the candidates. We use non-parallel corpus data in the two languages on the web and employ one of the two methods which we have developed. In the first method, we view the problem as that of classification and employ an ensemble of Naïve Bayesian Classifiers constructed with the EM Algorithm. We will use ‘EM-NBC-Ensemble’ to denote this method, hereafter. In the second method, we view the problem as that of calculating similarities between context vectors and use TF-IDF vectors also constructed with the EM Algorithm. We will use ‘EM-TF-IDF’ to denote this method. Experimental results indicate that our method is very effective, and the coverage and top 3 accuracy of translation at the final stage are 91.4% and 79.8%, respectively. The results are significantly better than those of the baseline methods relying on existing technologies. The higher performance of our method can be attributed to the enormity of the web data used and the employment of the EM Algorithm. 2. Related Work 2.1 Translation with Non-parallel Corpora A straightforward approach to word or phrase translation is to perform the task by using parallel bilingual corpora (e.g., Brown et al, 1993). Parallel corpora are, however, difficult to obtain in …

Content truncated to 500 words