Paraphrasing Out-of-Vocabulary Words with Word Embeddings and Semantic Lexicons for Low Resource Statistical Machine Translation

Paraphrasing Out-of-Vocabulary Words with Word Embeddings and Semantic Lexicons for Low Resource Statistical Machine Translation Chenhui Chu1 and Sadao Kurohashi2 1Japan Science and Technology Agency 2Graduate School of Informatics, Kyoto University E-mail: chu@pa.jst.jp, kuro@i.kyoto-u.ac.jp Abstract Out-of-vocabulary (OOV) word is a crucial problem in statistical machine translation (SMT) with low resources. OOV paraphrasing that augments the translation model for the OOV words by using the translation knowledge of their paraphrases has been proposed to address the OOV problem. In this paper, we propose using word embeddings and semantic lexicons for OOV paraphrasing. Experiments conducted on a low resource setting of the OLYMPICS task of IWSLT 2012 verify the effectiveness of our proposed method. Keywords: Paraphrasing, Out-of-Vocabulary Word, Word Embedding, Semantic Lexicon 1. Introduction In statistical machine translation (SMT) (Koehn et al., 2003), because translation knowledge is acquired from par- allel data, the quality and quantity of parallel data are cru- cial. However, except for a few language pairs, such as English-French, English-Arabic, English-Chinese and sev- eral European language pairs, parallel data remains a scarce resource. Moreover, even for these language pairs, the available domains are limited. The scarceness of parallel corpora makes the coverage of the translation model low, which leads to high out-of- vocabulary (OOV) word rates when conducting translation (Callison-Burch et al., 2006). Even we have parallel cor- pora in sufficient size in one domain, this OOV problem occurs when the domain shifts. Irvine et al. (2013a) showed that SMT performance decreases significantly when using a system trained on one domain to translate texts in different domains mainly because of OOVs. As one of the ways to address the OOV problem, para- phrasing has been proposed (Callison-Burch et al., 2006; Marton et al., 2009; Razmara et al., 2013). That is aug- menting the translation model for the OOV words by using the translation knowledge of their paraphrases in the trans- lation model. Previous studies use paraphrases generated by bilingual pivoting (Callison-Burch et al., 2006), distri- butional similarity (Marton et al., 2009), and graph propa- gation (Razmara et al., 2013), which suffer from high com- putational complexity. In this study, we propose using word embeddings (Mikolov et al., 2013) to address this problem. We also propose using semantic lexicons including Word- Net (Miller, 1995), FrameNet (Baker et al., 1998), and the Paraphrase Database (PPDB) (Ganitkevitch et al., 2013) for paraphrasing. In addition, we apply a method to combine these two types of paraphrases (Faruqui et al., 2015), which achieves further improvements in SMT. 2. Paraphrasing Out-of-Vocabulary Words In this paper, we study on phrase based SMT (Koehn et al., 2003). Figure 1 shows an overview of our proposed method. We first construct a phrase table based on unsu- pervised word alignments, containing phrase pairs together with their feature scores. From the development and test !"#"$%#&'($)) *"+,"+(! -.)///)0.)///),1-./0.2)$031-./0.2)4)) -5)///)05)///),1-5/052)$031-5/052)4)) -6)///)06)///),1-6/062)$031-6/062)4)) """) 7(+($$0$)*"+,'8) 9"+:5;0*) .).) .)) """) <0;)80=) >08=)80=) ??@)) 03=+(*A"#) ??@.)///)0.)///).).)4)) ??@5)///)05)///).).)4)) 8%B1??@.C)-.2) 8%B1??@5C)-52) ??@.) ??@5) """) D0B(#A*) $03%*"#! E"+:) 0BF0::%#&! """) """) Figure 1: Overview of our proposed method. sets, …

Content truncated to 500 words