Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3790–3806 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3790 Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution Ryuto Konno 1∗ Shun Kiyono 2,3 Yuichiroh Matsubayashi 3,2 Hiroki Ouchi 4,2 Kentaro Inui 3,2 1Recruit Co., Ltd. 2RIKEN 3Tohoku University 4Nara Institute of Science and Technology ryuto_konno@r.recruit.co.jp shun.kiyono@riken.jp hiroki.ouchi@is.naist.jpy.m@tohoku.ac.jp inui@tohoku.ac.jp Abstract Masked language models (MLMs) have con- tributed to drastic performance improvements with regard to zero anaphora resolution (ZAR). To further improve this approach, in this study, we made two proposals. The first is a new pre- training task that trains MLMs on anaphoric relations with explicit supervision, and the second proposal is a new finetuning method that remedies a notorious issue, the pretrain- finetune discrepancy. Our experiments on Japanese ZAR demonstrated that our two pro- posals boost the state-of-the-art performance, and our detailed analysis provides new insights on the remaining challenges. 1 Introduction In pronoun-dropping languages such as Japanese and Chinese, the semantic arguments of predicates can be omitted from sentences. As shown in Fig- ure 1, the semantic subject of the predicate used is omitted and represented by φ, which is called zero pronoun. This pronoun refers to the criminal in the first sentence. This way, the task of recogniz- ing the antecedents of zero pronouns is called zero anaphora resolution (ZAR). This study focuses on Japanese ZAR. ZAR is a challenging task because it requires reasoning with commonsense knowledge about the semantic associations between zero pronouns and the local contexts of their preceding antecedents. As shown in Figure 1, to identify the omitted se- mantic subject of used, the model should know the semantic relationship between the criminal’s weapon and a hammer, namely, a hammer is likely to be used as a weapon for murder and thus was used by the criminal, is crucial. We hereinafter refer to such knowledge as anaphoric relational knowledge. A conventional approach to acquire anaphoric re- lational knowledge is to collect predicate–argument *Work done while at Tohoku University. It seems that ! used a hammer. 犯人の criminal-GEN 使用した used The criminal’s weapon was found in the victim’s room. !-NOM !-NOM 凶器が weapon-NOM 見つかった。 was found. ハンマーを hammer-ACC 模様。 seem. 被害者の victim-GEN 部屋 room から from refer to Figure 1: Example of argument omission in Japanese. pairs from large-scale raw corpora and then, use them as features (Sasano et al., 2008; Sasano and Kurohashi, 2011; Yamashiro et al., 2018), or us- ing selectional preference probability (Shibata and Kurohashi, 2018) in machine learning models. A modern approach is to use masked language mod- els (MLMs) (Devlin et al., 2019), which is effec- tive in implicitly capturing anaphoric relational knowledge. In fact, recent studies used pretrained MLMs and achieved drastic performance improve- ments in the tasks that require anaphoric relational knowledge, including Japanese ZAR (Joshi et al., 2019; Aloraini and Poesio, 2020; Song et al., 2020; Konno et al., 2020). To get more out of MLMs, in this paper, we propose a new …

Content truncated to 500 words