Semantic-aware Chinese Zero Pronoun Resolution with Pre-trained Semantic Dependency Parser

Semantic-aware Chinese Zero Pronoun Resolution with Pre-trained Semantic Dependency Parser Lanqiu Zhang Beijing Language and Culture University zhang lanqiu@163.com Zizhuo Shen Beijing Language and Culture University blcushzz@gmail.com Yanqiu Shao� Beijing Language and Culture University yqshao163@163.com Abstract Deep learning-based Chinese zero pronoun resolution model has achieved better performance than traditional machine learning-based model. However, the existing work related to Chinese zero pronoun resolution has not yet well integrated linguistic information into the deep learning- based Chinese zero pronoun resolution model.This paper adopts the idea based on the pre-trained model, and integrates the semantic representations in the pre-trained Chinese semantic depen- dency graph parser into the Chinese zero pronoun resolution model. The experimental results on OntoNotes-5.0 dataset show that our proposed Chinese zero pronoun resolution model with pre- trained Chinese semantic dependency parser improves the F-score by 0.4% compared with our baseline model, and obtains better results than other deep learning-based Chinese zero pronoun resolution models. In addition, we integrate the BERT representations into our model so that the performance of our model was improved by 0.7% compared with our baseline model. 1 Introduction Chinese zero pronoun resolution is a special task of coreference resolution (Zhao and Ng, 2007). Its purpose is to find the real referent of the omitted parts with syntactic functions in the text. These omitted parts are usually called zero pronouns, and their real referents are called antecedents. Below is a sentence with zero pronouns: [·]ƒcvkf`L[¨]§[∗pro1∗]f`[∗pro∗2]´‡kâu�{å"(I have not heard of [her] before, [∗pro1∗] heard that [∗pro2∗] is a talented beauty.) In this example, the referent of zero pronoun ∗pro1∗ is /·/I0, and the referent of zero pronoun ∗pro2∗ is /¨/her0. Since the zero pronoun is not a real word in the text, its resolution is much more difficult than that of the overt pronoun. The existence of zero pronouns poses challenges for machines to automatically understand text. The existing Chinese zero pronoun resolution models with better performance usually adopt the method of deep learning (Chen and Ng, 2016);(Liu et al., 2017);(Yin et al., 2017);(Yin et al., 2018a);(Yin et al., 2018b). The deep learning-based methods can make the model automatically extract the task- related distributed representations through end-to-end training, thereby avoiding the problem that tradi- tional machine learning-based methods rely heavily on artificially designed feature templates (Chen and Ng, 2016). However, it is difficult for deep learning-based models to encode effective syntactic, semantic and other linguistic information only through end-to-end training. Many deep learning-based Chinese zero pronoun resolution models still use syntactic features extracted from the syntactic parsing tree as a supplement to distributed representations. Intuitively, semantic information as a higher level linguistic information is also very important to the Chinese zero pronoun resolution task, however few studies have attempted to integrate semantic information into the Chinese zero pronoun resolution model. Therefore, how to effectively integrate semantic information into the Chinese zero pronoun resolution model is a challenging problem. With the development of semantic parsing, the performance of some sentence-level semantic parsers have made c⃝2020 China National Conference on …

Content truncated to 500 words