Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4633 - 4646 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction Kuan-Hao Huang∗† I-Hung Hsu∗‡ Premkumar Natarajan‡ Kai-Wei Chang† Nanyun Peng†‡ †Computer Science Department, University of California, Los Angeles ‡Information Science Institute, University of Southern California {khhuang, kwchang, violetpeng}@cs.ucla.edu {ihunghsu, pnataraj}@isi.edu Abstract We present a study on leveraging multilin- gual pre-trained generative language models for zero-shot cross-lingual event argument ex- traction (EAE). By formulating EAE as a lan- guage generation task, our method effectively encodes event structures and captures the de- pendencies between arguments. We design language-agnostic templates to represent the event argument structures, which are compat- ible with any language, hence facilitating the cross-lingual transfer. Our proposed model finetunes multilingual pre-trained generative language models to generate sentences that fill in the language-agnostic template with argu- ments extracted from the input passage. The model is trained on source languages and is then directly applied to target languages for event argument extraction. Experiments demonstrate that the proposed model outper- forms the current state-of-the-art models on zero-shot cross-lingual EAE. Comprehensive studies and error analyses are presented to bet- ter understand the advantages and the current limitations of using generative language mod- els for zero-shot cross-lingual transfer EAE. 1 Introduction Event argument extraction (EAE) aims to recog- nize the entities serving as event arguments and identify their corresponding roles. As illustrated by the English example in Figure 1, given a trig- ger word “destroyed” for a Conflict:Attack event, an event argument extractor is expected to iden- tify “commando”, “Iraq”, and “post” as the event arguments and predict their roles as “At- tacker”, “Place”, and “Target”, respectively. Zero-shot cross-lingual EAE has attracted con- siderable attention since it eliminates the require- ment of labeled data for constructing EAE models in low-resource languages (Subburathinam et al., 2019; Ahmad et al., 2021; Nguyen and Nguyen, ∗The authors contribute equally. Attacker Place Target Attacker Target 接近高级军官的消息灵通人士 说，南斯拉夫 军队 不会离 开军营去干涉 反对派 起义。 Australian commandos , who have been operating deep in Iraq , destroyed a command and control post and killed a number of soldiers. Figure 1: An illustration of cross-lingual event ar- gument extraction. Given sentences in arbitrary lan- guages and their event triggers (destroyed and 起义), the model needs to identify arguments (commando, Iraq and post v.s. 军队, and 反对派) and their cor- responding roles (Attacker, Target, and Place). 2021). In this setting, the model is trained on the ex- amples in the source languages and directly tested on the instances in the target languages. Recently, generation-based models1 have shown strong performances on monolingual structured pre- diction tasks (Yan et al., 2021; Huang et al., 2021b; Paolini et al., 2021), including EAE (Li et al., 2021; Hsu et al., 2021). These works fine-tune pre-trained generative language models to generate outputs fol- lowing designed templates such that the final pre- dictions can be easily decoded from the outputs. Compared to …

Content truncated to 500 words