Coupling Context Modeling with Zero Pronoun Recovering for Document-Level Natural Language Generation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2530–2540 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2530 Coupling Context Modeling with Zero Pronoun Recovering for Document-Level Natural Language Generation Xin Tan, Longyin Zhang, and Guodong Zhou∗ School of Computer Science and Technology, Soochow University, China {xtan9,lyzhang9}@stu.suda.edu.cn gdzhou@suda.edu.cn Abstract Natural language generation (NLG) tasks on pro-drop languages are known to suffer from zero pronoun (ZP) problems, and the prob- lems remain challenging due to the scarcity of ZP-annotated NLG corpora. In this case, we propose a highly adaptive two-stage approach to couple context modeling with ZP recover- ing to mitigate the ZP problem in NLG tasks. Notably, we frame the recovery process in a task-supervised fashion where the ZP repre- sentation recovering capability is learned dur- ing the NLG task learning process, thus our method does not require NLG corpora anno- tated with ZPs. For system enhancement, we learn an adversarial bot to adjust our model out- puts to alleviate the error propagation caused by mis-recovered ZPs. Experiments on three document-level NLG tasks, i.e., machine trans- lation, question answering, and summarization, show that our approach can improve the perfor- mance to a great extent, and the improvement on pronoun translation is very impressive. 1 Introduction For a long time, natural language generation (NLG) has attracted a lot of attention for its importance in serving human life. In the literature, although vari- ous studies have been done to bridge the discrep- ancy between human and machine, document-level NLG (D-NLG) tasks still suffer from cohesion is- sues caused by zero pronoun (ZP). As a discourse phenomenon where pronouns can be omitted when they are pragmatically or grammatically inferable from context (Li and Thompson, 1979), zero pro- noun appears frequently in pro-drop languages like Chinese, Spanish, etc. Taking the Chinese TED corpus as an example, according to our statistics, in sentences with an average length of 18, each sentence will omit around 0.5 pronouns. Facing this problem, lots of attention has been paid to ZP resolution in the past decade (Zhao and Ng, 2007; ∗Corresponding author Kong and Zhou, 2010; Yin et al., 2018; Zhang et al., 2019a; Song et al., 2020), and these studies have achieved certain success. Nevertheless, due to the lack of NLG corpora annotated with ZPs, the problem of zero pronoun remains challenging in document-level NLG tasks. Recently, more and more researchers turn to ZP resolution in NLG tasks (Rao et al., 2015; Wang et al., 2016, 2018a,b, 2019). Among these studies, one line explicitly deals with the ZP problem by recovering dropped pronouns through either anno- tated corpora (Wang et al., 2016, 2018a,b, 2019; Zhang et al., 2019c) or pre-trained ZP resolution systems (Taira et al., 2012; Xiang et al., 2013). An- other line indirectly deals with the ZP problem by producing better discourse cohesion through doc- ument context modeling (Zhang et al., 2018; Mi- culicich et al., 2018; Tan et al., 2019; Wong et al., 2020). Although the above studies have made …

Content truncated to 500 words