Measure Word Generation for English-Chinese SMT Systems

Proceedings of ACL-08: HLT, pages 89–96, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Measure Word Generation for English-Chinese SMT Systems Dongdong Zhang1, Mu Li1, Nan Duan2, Chi-Ho Li1, Ming Zhou1 1Microsoft Research Asia 2Tianjin University Beijing, China Tianjin, China {dozhang,muli,v-naduan,chl,mingzhou}@microsoft.com Abstract Measure words in Chinese are used to indi- cate the count of nouns. Conventional sta- tistical machine translation (SMT) systems do not perform well on measure word generation due to data sparseness and the potential long distance dependency between measure words and their corresponding head words. In this paper, we propose a statistical model to gen- erate appropriate measure words of nouns for an English-to-Chinese SMT system. We mod- el the probability of measure word generation by utilizing lexical and syntactic knowledge from both source and target sentences. Our model works as a post-processing procedure over output of statistical machine translation systems, and can work with any SMT system. Experimental results show our method can achieve high precision and recall in measure word generation. 1 Introduction In linguistics, measure words (MW) are words or morphemes used in combination with numerals or demonstrative pronouns to indicate the count of nouns1, which are often referred to as head words (HW). Chinese measure words are grammatical units and occur quite often in real text. According to our survey on the measure word distribution in the Chinese Penn Treebank and the test datasets distri- buted by Linguistic Data Consortium (LDC) for Chinese-to-English machine translation evaluation, the average occurrence is 0.505 and 0.319 measure 1 The uncommon cases of verbs are not considered. words per sentence respectively. Unlike in Chinese, there is no special set of measure words in English. Measure words are usually used for mass nouns and any semantically appropriate nouns can func- tion as the measure words. For example, in the phrase three bottles of water, the word bottles acts as a measure word. Countable nouns are almost never modified by measure words2. Numerals and indefinite articles are directly followed by counta- ble nouns to denote the quantity of objects. Therefore, in the English-to-Chinese machine translation task we need to take additional efforts to generate the missing measure words in Chinese. For example, when translating the English phrase three books into the Chinese phrases “三本书”, where three corresponds to the numeral “三” and books corresponds to the noun “书”, the Chinese measure word “本” should be generated between the numeral and the noun. In most statistical machine translation (SMT) models (Och et al., 2004; Koehn et al., 2003; Chiang, 2005), some of measure words can be generated without modification or additional processing. For example, in above translation, the phrase translation table may suggest the word three be translated into “三”, “三本”, “三只”, etc, and the word books into “书”, “书本”, “名册” (scroll), etc. Then the SMT model selects the most likely combination “三本书” as the final translation re- sult. In this example, a measure word candidate set consisting of “本” and “只” can be generated by bilingual phrases …

Content truncated to 500 words