ZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5429–5434 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5429 ZPR2: Joint Zero Pronoun Recovery and Resolution using Multi-Task Learning and BERT Linfeng Song1, Kun Xu1, Yue Zhang2,3, Jianshu Chen1 and Dong Yu1 1Tencent AI Lab, Bellevue, WA, USA 2School of Engineering, Westlake University, China 3Institute of Advanced Technology, Westlake Institute for Advanced Study, China Abstract Zero pronoun recovery and resolution aim at recovering the dropped pronoun and pointing out its anaphoric mentions, respectively. We propose to better explore their interaction by solving both tasks together, while the previous work treats them separately. For zero pronoun resolution, we study this task in a more real- istic setting, where no parsing trees or only automatic trees are available, while most pre- vious work assumes gold trees. Experiments on two benchmarks show that joint modeling significantly outperforms our baseline that al- ready beats the previous state of the arts. Our code is available at https://github.com/ freesunshine0316/lab-zp-joint. 1 Introduction Zero pronoun (ZP) is a linguistic phenomenon where a pronoun is dropped for simplicity. Fig- ure 1 shows an example, where two pronouns at positions φ1 and φ2 are omitted. They both refer to “fπ (The police)” in the sentence beginning and their original form is “÷Ï (they)”. The sit- uation of dropping pronouns happens in most lan- guages. While this phenomenon is not frequent in non-pro-drop languages, such as English, it is extremely severe for pro-drop languages, such as Chinese. In addition, dropped pronouns happens more frequently in conversations than in news. Our preliminary statistics of Chinese shows that 59.2% pronouns are dropped in a corpus of casual dia- logues domain, while the number is just 41.6% in another data of broadcast news. In NLP, dropped pronouns can cause loss of im- portant information, such as the subject or object of the central predicate in a sentence, introducing ambiguity to applications such as machine transla- tion (Nakaiwa and Shirai, 1996; Wang et al., 2016; Takeno et al., 2016), question answering (Choi et al., 2018; Reddy et al., 2019; Sun et al., 2019; [ fπ ] 怀ë Ÿ / 一w —™ Hˆ ， φ1 ⌃ ™∞ å Æ⇧ §送 ⇥à φ2  ⇧⌃ H≈ ⇥ [ The police ] suspected that this is a criminal case about illegal guns , φ1 brought the guns and bags to the city φ2 to deal with the case . Figure 1: An zero pronoun example and its English translation, where φ1 and φ2 are zero pronouns point- ing to the span in square brackets. Chen and Choi, 2016) and dialogue understanding (Chen et al., 2017; Rolih, 2018). As a result, zero pronouns have recently received much research at- tention (Liu et al., 2017; Yin et al., 2018a,b). We study Chinese zero pronoun in dialogue settings. There are two long-existing tasks namely zero pronoun recovery, which aims at recovering the original pronoun (such as “÷ (he)” and “y (she)”), and zero pronoun resolution, …

Content truncated to 500 words