One Model to Learn Both: Zero Pronoun Prediction and Translation

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 921–930, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 921 One Model to Learn Both: Zero Pronoun Prediction and Translation Longyue Wang Tencent AI Lab vinnylywang@tencent.com Zhaopeng Tu Tencent AI Lab zptu@tencent.com Xing Wang Tencent AI Lab brightxwang@tencent.com Shuming Shi Tencent AI Lab shumingshi@tencent.com Abstract Zero pronouns (ZPs) are frequently omitted in pro-drop languages, but should be recalled in non-pro-drop languages. This discourse phe- nomenon poses a significant challenge for ma- chine translation (MT) when translating texts from pro-drop to non-pro-drop languages. In this paper, we propose a unified and discourse- aware ZP translation approach for neural MT models. Specifically, we jointly learn to predict and translate ZPs in an end-to-end manner, allowing both components to inter- act with each other. In addition, we em- ploy hierarchical neural networks to exploit discourse-level context, which is beneficial for ZP prediction and thus translation. Ex- perimental results on both Chinese⇒English and Japanese⇒English data show that our ap- proach significantly and accumulatively im- proves both translation performance and ZP prediction accuracy over not only baseline but also previous works using external ZP predic- tion models. Extensive analyses confirm that the performance improvement comes from the alleviation of different kinds of errors espe- cially caused by subjective ZPs. 1 Introduction Zero anaphora is a discourse phenomenon, where pronouns can be omitted when they are pragmat- ically or grammatically inferable from intra- and inter-sentential context (Li and Thomson, 1979). However, translating such implicit information (i.e. zero pronoun, ZP) poses various difficulties for machine translation (MT) in terms of com- pleteness and correctness. Although neural mod- els are getting better at learning representations, it is still difficult to implicitly learn complex ZPs in a general model. Actually, ZP prediction and trans- lation need to not only understand the semantics or intentions of a single sentence, but also utilize its discourse-level context. Two technological advances in the field of ZP and MT, have seen vast progress over the last decades, but they have been developed very much in isolation. Early studies (Chung and Gildea, 2010; Le Nagard and Koehn, 2010; Xiang et al., 2013) fed MT systems with the results of ZP pre- diction models, which are trained on a small-scale and non-homologous data compared to MT mod- els. To narrow the data-level gap, Wang et al. (2016) proposed an automatic method to annotate ZPs by utilizing the parallel corpus of MT. The ho- mologous data for both ZP prediction and transla- tion leads to significant improvements on transla- tion performances for both statistical MT (Wang et al., 2016) and neural MT models (Wang et al., 2018a). However, such approaches still require external ZP prediction models, which have a low accuracy of 66%. The numerous errors of ZP prediction errors will be propagated to transla- tion models, which leads to new translation prob- lems. In addition, relying on external ZP predic- …

Content truncated to 500 words