Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 102–111 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1010 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 102–111 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1010 Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution Ting Liu†, Yiming Cui‡, Qingyu Yin†, Weinan Zhang†, Shijin Wang‡ and Guoping Hu‡ †Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, Harbin, China ‡iFLYTEK Research, Beijing, China †{tliu,qyyin,wnzhang}@ir.hit.edu.cn ‡{ymcui,sjwang3,gphu}@iflytek.com Abstract Most existing approaches for zero pronoun resolution are heavily relying on annotated data, which is often released by shared task organizers. Therefore, the lack of annotated data becomes a major obstacle in the progress of zero pronoun resolution task. Also, it is expensive to spend man- power on labeling the data for better per- formance. To alleviate the problem above, in this paper, we propose a simple but novel approach to automatically generate large-scale pseudo training data for zero pronoun resolution. Furthermore, we suc- cessfully transfer the cloze-style reading comprehension neural network model into zero pronoun resolution task and propose a two-step training mechanism to over- come the gap between the pseudo training data and the real one. Experimental re- sults show that the proposed approach sig- nificantly outperforms the state-of-the-art systems with an absolute improvements of 3.1% F-score on OntoNotes 5.0 data. 1 Introduction Previous works on zero pronoun (ZP) resolution mainly focused on the supervised learning ap- proaches (Han, 2006; Zhao and Ng, 2007; Iida et al., 2007; Kong and Zhou, 2010; Iida and Poe- sio, 2011; Chen and Ng, 2013). However, a ma- jor obstacle for training the supervised learning models for ZP resolution is the lack of anno- tated data. An important step is to organize the shared task on anaphora and coreference resolu- tion, such as the ACE evaluations, SemEval-2010 shared task on Coreference Resolution in Multiple Languages (Marta Recasens, 2010) and CoNLL- 2012 shared task on Modeling Multilingual Unre- stricted Coreference in OntoNotes (Sameer Prad- han, 2012). Following these shared tasks, the an- notated evaluation data can be released for the fol- lowing researches. Despite the success and con- tributions of these shared tasks, it still faces the challenge of spending manpower on labeling the extended data for better training performance and domain adaptation. To address the problem above, in this paper, we propose a simple but novel approach to automati- cally generate large-scale pseudo training data for zero pronoun resolution. Inspired by data genera- tion on cloze-style reading comprehension, we can treat the zero pronoun resolution task as a special case of reading comprehension problem. So we can adopt similar data generation methods of read- ing comprehension to the zero pronoun resolution task. For the noun or pronoun in the document, which has the frequency equal to or greater than 2, we randomly choose one position where the noun or pronoun is …

Content truncated to 500 words