Zero Pronoun Resolution with Attention-based Neural Network

Proceedings of the 27th International Conference on Computational Linguistics, pages 13–23 Santa Fe, New Mexico, USA, August 20-26, 2018. 13 Zero Pronoun Resolution with Attention-based Neural Network Qingyu Yin♯, Yu Zhang♯∗, Weinan Zhang♯, Ting Liu♯, William Yang Wang♭ ♯Harbin Institute of Technology, China ♭University of California, Santa Barbara, USA {qyyin, yzhang, wnzhang, tliu}@ir.hit.edu.cn william@cs.ucsb.edu Abstract Recent neural network methods for zero pronoun resolution explore multiple models for generat- ing representation vectors for zero pronouns and their candidate antecedents. Typically, contex- tual information is utilized to encode the zero pronouns since they are simply gaps that contain no actual content. To better utilize contexts of the zero pronouns, we here introduce the self- attention mechanism for encoding zero pronouns. With the help of the multiple hops of attention, our model is able to focus on some informative parts of the associated texts and therefore pro- duces an efficient way of encoding the zero pronouns. In addition, an attention-based recurrent neural network is proposed for encoding candidate antecedents by their contents. Experiment results are encouraging: our proposed attention-based model gains the best performance on the Chinese portion of the OntoNotes corpus, substantially surpasses existing Chinese zero pronoun resolution baseline systems. Title and Abstract in Chinese 基于注意力神经网络模型的零指代消解研究 传统的关于零指代的方法提出了多种关于先行语和零代词的表示模型。这些模型中，研 究者们用零代词的上下文信息来帮助表示缺省的信息。为了更好的帮助建模零代词，我 们提出了一种基于注意力机制的神经网络模型，通过注意力模型来获取更有表示性信 息的上下文信息。实验结果表明：我们的方法能够有效提升效果，并在中文OntoNotes 5.0数据集上取得了最好的结果，超越了现有的基准系统。 1 Introduction In natural languages, expressions that can be deduced contextually by people are frequently omitted in texts. This is special the case in pro-dropped languages, such as Chinese, where a kind of anaphoric expression is frequently eliminated. A zero pronoun is a gap in the sentence that is found when a pho- netically null form is used to refer to a real-world entity (Chen and Ng, 2016). We here show a case of zero pronouns from the OntoNotes-5.0 dataset. 这 次 地震 φ1 有 一些 房屋 塌 的 , 这 里面 如果 有 建房 的 质量 问题 ， φ2 是 要 追究 责任 的 。 In this earthquck φ1 some rooms collapsed, if there exsit some room quality issues, φ2 will need to call to account. We use φ to represent the zero pronouns in this example. Among these zero anaphoras, we can assign the mention “政府/the government” that appears in leading text, to be the antecedent of φ2 while there are no such mentions for φ1. Hence, φ2 is an anaphoric zero pronoun, and φ1 is the un-anaphoric case. ∗Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ 14 With the fact that zero pronouns are gaps that have no text, it is almost impracticable to representant the zero pronouns by themselves. This issue has received increasingly attention. In recent time, deep neural network methods for Chinese zero pronoun resolution (Chen and Ng, 2016; Yin et al., 2017a; Yin et al., 2017b) have been proposed and are intended to encode zero pronouns into the vector-space semantic by additional elements. Chen and Ng (2016) propose a neural network model that learns to encode the anaphoric zero pronoun by using the leading …

Content truncated to 500 words