Chinese Zero Pronoun Resolution with Deep Neural Networks

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 778–788, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Chinese Zero Pronoun Resolution with Deep Neural Networks Chen Chen and Vincent Ng Human Language Technology Research Institute University of Texas at Dallas Richardson, TX 75083-0688 {yzcchen,vince}@hlt.utdallas.edu Abstract While unsupervised anaphoric zero pro- noun (AZP) resolvers have recently been shown to rival their supervised counter- parts in performance, it is relatively diffi- cult to scale them up to reach the next level of performance due to the large amount of feature engineering efforts involved and their ineffectiveness in exploiting lexical features. To address these weaknesses, we propose a supervised approach to AZP resolution based on deep neural networks, taking advantage of their ability to learn useful task-specific representations and ef- fectively exploit lexical features via word embeddings. Our approach achieves state- of-the-art performance when resolving the Chinese AZPs in the OntoNotes corpus. 1 Introduction A zero pronoun (ZP) is a gap in a sentence that is found when a phonetically null form is used to refer to a real-world entity. An anaphoric zero pro- noun (AZP) is a ZP that corefers with one or more preceding mentions in the associated text. Below is an example taken from the Chinese Treebank (CTB), where the ZP (denoted as *pro*) refers to 俄罗斯 (Russia). [俄罗斯] 作为米洛舍夫维奇一贯的支持者， *pro* 曾经提出调停这场政治危机。 ([Russia] is a consistent supporter of Milošević, *pro* has proposed to mediate the political crisis.) As we can see, ZPs lack grammatical attributes that are useful for overt pronoun resolution such as number and gender. This makes ZP resolution more challenging than overt pronoun resolution. Automatic ZP resolution is typically composed of two steps. The first step, AZP identification, in- volves extracting ZPs that are anaphoric. The sec- ond step, AZP resolution, aims to identify an an- tecedent of an AZP. State-of-the-art ZP resolvers have tackled both of these steps in a supervised manner, training one classifier for AZP identifica- tion and another for AZP resolution (e.g., Zhao and Ng (2007), Kong and Zhou (2010)). More recently, Chen and Ng (2014b; 2015) have proposed unsupervised probabilistic AZP resolu- tion models (henceforth the CN14 model and the CN15 model, respectively) that rival their super- vised counterparts in performance. An appeal- ing aspect of these unsupervised models is that their language-independent generative process en- ables them to be applied to languages where data annotated with ZP links are not readily avail- able. Though achieving state-of-the-art perfor- mance, these models have several weaknesses. First, a lot of manual efforts need to be spent on engineering the features for generative proba- bilistic models, as these models are sensitive to the choice of features. For instance, having features that are (partially) dependent on each other could harm model performance. Second, in the absence of labeled data, it is difficult, though not impos- sible, for these models to profitably employ lexi- cal features (e.g., word pairs, syntactic patterns in- volving words), as determining which lexical fea- tures …

Content truncated to 500 words