Learning Semantic Representations for Nonterminals in Hierarchical Phrase-Based Translation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1391–1400, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Learning Semantic Representations for Nonterminals in Hierarchical Phrase-Based Translation Xing Wang, Deyi Xiong∗and Min Zhang Soochow University, Suzhou, China 215006 xingwsuda@gmail.com, {dyxiong, minzhang}@suda.edu.cn Abstract In hierarchical phrase-based translation, coarse-grained nonterminal Xs may gen- erate inappropriate translations due to the lack of sufficient information for phrasal substitution. In this paper we propose a framework to refine nonterminals in hier- archical translation rules with real-valued semantic representations. The semantic representations are learned via a weighted mean value and a minimum distance method using phrase vector representa- tions obtained from large scale monolin- gual corpus. Based on the learned se- mantic vectors, we build a semantic non- terminal refinement model to measure se- mantic similarities between phrasal sub- stitutions and nonterminal Xs in transla- tion rules. Experiment results on Chinese- English translation show that the proposed model significantly improves translation quality on NIST test sets. 1 Introduction Hierarchical phrase-based translation (Chiang, 2007) explores formal synchronous context free grammar (SCFG) rules for translation. Two types of nonterminal symbols are used in translation rules: nonterminal X in ordinary SCFG rules and nonterminal S in glue rules that are specially intro- duced to concatenate nonterminal Xs in a mono- tonic manner. The same generic symbol X for all ordinary nonterminals makes it difficult to distin- guish and select proper translation rules. In order to address this issue, researchers ei- ther use syntactic labels to annotate nontermi- nal Xs (Zollmann and Venugopal, 2006; Zoll- mann and Vogel, 2011; Li et al., 2012; Hanneman and Lavie, 2013), or employ syntactic information ∗Corresponding author from parse trees to refine nonterminals with real- valued vectors (Venugopal et al., 2009; Huang et al., 2013). In addition to syntactic knowledge, se- mantic structures are also leveraged to refine non- terminals (Gao and Vogel, 2011). All these efforts focus on incorporating linguistic knowledge into hierarchical translation rules. Unfortunately, syntactic or semantic parsers for many languages are not accessible due to the lack of labeled training data. In contrast, a large amount of unlabeled data are easily available. Therefore, can we mine syntactic or semantic properties for nonterminals from unlabeled data? Or can we exploit these data to refine nontermi- nals for SMT? Learning semantic representations for terminals (words, multi-word phrases or sentences) from un- labeled data has achieved substantial progress in recent years (Mitchell and Lapata, 2008; Turian et al., 2010; Socher et al., 2010; Mikolov et al., 2013c; Blunsom et al., 2014). These rep- resentations have been used successfully in var- ious NLP tasks. However, there is no attempt to learn semantic representations for nontermi- nals from unlabeled data. In this paper we pro- pose a framework to learn semantic representa- tions for nonterminal Xs in translation rules. Our framework is established on the basis of real- valued vector representations learned for multi- word phrases, which are substituted with nonter- minal Xs during hierarchical rule extraction. We propose …

Content truncated to 500 words