Enlisting the Ghost: Modeling Empty Categories for Machine Translation

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 822–831, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Enlisting the Ghost: Modeling Empty Categories for Machine Translation Bing Xiang IBM T. J. Watson Research Center 1101 Kitchawan Rd Yorktown Heights, NY 10598 bxiang@us.ibm.com Xiaoqiang Luo * Google Inc. 111 8th Ave New York, NY 10011 xql@google.com Bowen Zhou IBM T. J. Watson Research Center 1101 Kitchawan Rd Yorktown Heights, NY 10598 zhou@us.ibm.com Abstract Empty categories (EC) are artificial ele- ments in Penn Treebanks motivated by the government-binding (GB) theory to ex- plain certain language phenomena such as pro-drop. ECs are ubiquitous in languages like Chinese, but they are tacitly ignored in most machine translation (MT) work because of their elusive nature. In this paper we present a comprehensive treat- ment of ECs by first recovering them with a structured MaxEnt model with a rich set of syntactic and lexical features, and then incorporating the predicted ECs into a Chinese-to-English machine translation task through multiple approaches, includ- ing the extraction of EC-specific sparse features. We show that the recovered empty categories not only improve the word alignment quality, but also lead to significant improvements in a large-scale state-of-the-art syntactic MT system. 1 Introduction One of the key challenges in statistical machine translation (SMT) is to effectively model inher- ent differences between the source and the target language. Take the Chinese-English SMT as an example: it is non-trivial to produce correct pro- nouns on the target side when the source-side pro- noun is missing. In addition, the pro-drop prob- lem can also degrade the word alignment qual- ity in the training data. A sentence pair observed in the real data is shown in Figure 1 along with the word alignment obtained from an automatic word aligner, where the English subject pronoun * This work was done when the author was with IBM. “that” is missing on the Chinese side. Conse- quently, “that” is incorrectly aligned to the second to the last Chinese word “De”, due to their high co-occurrence frequency in the training data. If the dropped pronoun were recovered, “that” would have been aligned with the dropped-pro (cf. Fig- ure 3), which is a much more sensible alignment. Figure 1: Example of incorrect word alignment due to missing pronouns on the Chinese side. In order to account for certain language phe- nomena such as pro-drop and wh-movement, a set of special tokens, called empty categories (EC), are used in Penn Treebanks (Marcus et al., 1993; Bies and Maamouri, 2003; Xue et al., 2005). Since empty categories do not exist in the surface form of a language, they are often deemed elusive and recovering ECs is even figuratively called “chas- ing the ghost” (Yang and Xue, 2010). In this work we demonstrate that, with the avail- ability of large-scale EC annotations, it is feasi- ble to predict and recover ECs with high accu- racy. More importantly, with various approaches of modeling the recovered ECs in SMT, …

Content truncated to 500 words