Improving Statistical Machine Translation Accuracy Using Bilingual Lexicon Extractionwith Paraphrases

PACLIC 28 !262 Improving Statistical Machine Translation Accuracy Using Bilingual Lexicon Extraction with Paraphrases Chenhui Chu1,3, Toshiaki Nakazawa2, Sadao Kurohashi1 1Graduate School of Informatics, Kyoto University 2Japan Science and Technology Agency 3Japan Society for the Promotion of Science Research Fellow chu@nlp.ist.i.kyoto-u.ac.jp, nakazawa@pa.jst.jp, kuro@i.kyoto-u.ac.jp Abstract Statistical machine translation (SMT) suffers from the accuracy problem that the translation pairs and their feature scores in the transla- tion model can be inaccurate. The accuracy problem is caused by the quality of the unsu- pervised methods used for translation model learning. Previous studies propose estimating comparable features for the translation pairs in the translation model from comparable cor- pora, to improve the accuracy of the transla- tion model. Comparable feature estimation is based on bilingual lexicon extraction (BLE) technology. However, BLE suffers from the data sparseness problem, which makes the comparable features inaccurate. In this paper, we propose using paraphrases to address this problem. Paraphrases are used to smooth the vectors used in comparable feature estimation with BLE. In this way, we improve the qual- ity of comparable features, which can improve the accuracy of the translation model thus im- prove SMT performance. Experiments con- ducted on Chinese-English phrase-based SMT (PBSMT) verify the effectiveness of our pro- posed method. 1 Introduction In statistical machine translation (SMT) (Brown et al., 1993), the translation model is automatically learned form parallel corpora in an unsupervised way. The translation model contains translation pairs with their features scores. SMT suffers from the accuracy problem that the translation model may be inaccurate, meaning that the translation pairs and their features scores may be inaccurate. The accu- racy problem is caused by the quality of the unsu- pervised method used for translation model learning, which always correlates with the amount of parallel corpora. Increasing the amount of parallel corpora is a possible way to improve the accuracy, however parallel corpora remain a scarce resource for most language pairs and domains.1 Accuracy also can be improved by filtering out the noisy translation pairs from the translation model, however meanwhile we may lose some good translation pairs, thus the cov- erage of the translation model may decrease. A good solution to improve the accuracy while keeping the coverage is estimating new features for the transla- tion pairs from comparable corpora (which we call comparable features), to make the translation model more discriminative thus more accurate. Previous studies use bilingual lexicon extraction (BLE) technology to estimate comparable features (Klementiev et al., 2012; Irvine and Callison-Burch, 2013a). They extend traditional BLE that estimates similarity for bilingual word pairs on comparable corpora, to translation pairs in the translation model of SMT. The similarity scores of the translation pairs are used as comparable features. These compara- ble features are combined with the original features used in SMT, which can provide additional informa- tion to distinguish good and bad translation pairs. A major problem of previous studies is that they do not deal with the data sparseness problem that BLE suf- fers from. BLE uses vector representations for …

Content truncated to 500 words