Mining Parenthetical Translations from the Web by Word Alignment

Proceedings of ACL-08: HLT, pages 994–1002, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Mining Parenthetical Translations from the Web by Word Alignment Dekang Lin Shaojun Zhao† Benjamin Van Durme† Marius Paşca Google, Inc. University of Rochester University of Rochester Google, Inc. Mountain View Rochester Rochester Mountain View CA, 94043 NY, 14627 NY, 14627 CA, 94043 lindek@google.com zhao@cs.rochester.edu vandurme@cs.rochester.edu mars@google.com Abstract Documents in languages such as Chinese, Japanese and Korean sometimes annotate terms with their translations in English inside a pair of parentheses. We present a method to extract such translations from a large collec- tion of web documents by building a partially parallel corpus and use a word alignment al- gorithm to identify the terms being translated. The method is able to generalize across the translations for different terms and can relia- bly extract translations that occurred only once in the entire web. Our experiment on Chinese web pages produced more than 26 million pairs of translations, which is over two orders of magnitude more than previous re- sults. We show that the addition of the ex- tracted translation pairs as training data provides significant increase in the BLEU score for a statistical machine translation sys- tem. 1 Introduction In natural language documents, a term (word or phrase) is sometimes followed by its translation in another language in a pair of parentheses. We call these parenthetical translations. The following examples are from Chinese web pages (we added underlines to indicate what is being translated): (1) 美国智库布鲁金斯学会（Brookings Institution）专研 跨大西洋恐怖主义的美欧中心研究部主任杰若米·夏皮 罗（Jeremy Shapiro）却认为，... (2) 消化性溃疡的症状往往与消化不良（indigestion），胃 炎（gastritis）等其他胃部疾病症状相似. (3) 殊不知美国是不会接受（not going to fly）这一想法的 (4) …当是一次式时，叫线性规划(linear programming). †Contributions made during an internship at Google The parenthetically translated terms are typically new words, technical terminologies, idioms, prod- ucts, titles of movies, books, songs, and names of persons, organizations locations, etc. Commonly, an author might use such a parenthetical when a given term has no standard translation (or translit- eration), and does not appear in conventional dic- tionaries. That is, an author might expect a term to be an out-of-vocabulary item for the target reader, and thus helpfully provides a reference translation in situ. For example, in (1), the name Shapiro was transliterated as 夏皮罗. The name has many other transliterations in web documents, such as 夏皮洛, 夏比洛, 夏布洛, 夏皮羅, 沙皮罗, 夏皮若, 夏庇罗, 夏皮諾, 夏畢洛, 夏比羅, 夏比罗, 夏普羅, 夏批羅, 夏批罗, 夏彼羅, 夏彼罗, 夏培洛, 夏卜尔, 夏匹若 ..., where the three Chinese characters corresponds to the three sylla- bles in Sha-pi-ro respectively. Each syllable may be mapped into different characters: 'Sha' into 夏 or 沙, 'pi' into 皮, 比, 批, and 'ro' into 罗, 洛, 若, .... Variation is not limited to the effects of phonetic similarity. Story titles, for instance, are commonly translated semantically, often leading to a number of translations that have similar meaning, yet differ greatly in lexicographic form. For example, while the movie title Syriana is sometimes phonetically transliterated as 辛瑞那, 辛瑞纳, it may also be trans- lated semantically according to the plot of the movie, e.g., 迷中迷 (mystery in mystery), 实录 (real …

Content truncated to 500 words