Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction

Findings of the Association for Computational Linguistics: EACL 2023, pages 754–769 May 2-6, 2023 ©2023 Association for Computational Linguistics Bridging the Gap between Native Text and Translated Text through Adversarial Learning: A Case Study on Cross-Lingual Event Extraction Pengfei Yu1, Jonathan May2, Heng Ji1 1University of Illinois Urbana-Champaign 2University of South California {pengfei4,hengji}@illinois.edu jonmay@isi.edu Abstract Recent research in cross-lingual learning has found that combining large-scale pretrained multilingual language models with machine translation can yield good performance (Phang et al., 2020; Fang et al., 2021). We explore this idea for cross-lingual event extraction with a new model architecture that jointly encodes a source language input sentence with its trans- lation to the target language during training, and takes a target language sentence with its translation back to the source language as input during evaluation. However, we observe signif- icant representational gap between the native texts and translated texts, both in the source language and the target language. This repre- sentational gap undermines the effectiveness of cross-lingual transfer learning for event ex- traction with machine-translated data. In order to mitigate this problem, we propose an adver- sarial training framework that encourages the language model to produce more similar rep- resentations for the translated text and the na- tive text. To be specific, we train the language model such that its hidden representations are able to fool a jointly trained discriminator that distinguishes translated texts’ representations from native texts’ representations. We conduct experiments on cross-lingual event extraction across three languages. Results demonstrate that our proposed adversarial training can ef- fectively incorporate machine translation to im- prove event extraction, while simply adding machine-translated data yields unstable perfor- mance due to the representational gap.1 1 Introduction There are over 6,000 living languages in the world, and for many of them, too little appropriate data exists to build natural language processing (NLP) models. Cross-lingual learning has been proposed to leverage resources in data-rich languages to train NLP models for data-scarce languages (Ruder 1Code at https://github.com/Perfec-Yu/CrossIE et al., 2019). There are two main strategies for building cross-lingual models: (1) train models with multilingual language models and language- universal features that are transferable to the target language (Huang et al., 2019; Hsu et al., 2019; Hu et al., 2020a; Luo et al., 2020; Wei et al., 2021; Ouyang et al., 2021; Liu et al., 2019; Subburathi- nam et al., 2019; M’hamdi et al., 2019; Ahmad et al., 2021); (2) use machine translation models in a pipeline, either by transforming annotated train- ing data into the desired target language to build target-language models, or by translating data at in- ference time into the source language and applying source-language models (Cui et al., 2019; Hu et al., 2020a; Yarmohammadi et al., 2021). The first ap- proach relies on the quality of the constructed mul- tilingual semantic space; the discrepancy between source-language training data and target-language evaluation data may cause overfitting. The second approach does not require a perfect multilingual semantic space since models can be trained in a monolingual fashion, …

Content truncated to 500 words