Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5414–5426 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5414 Crosslingual Transfer Learning for Relation and Event Extraction via Word Category and Class Alignments Minh Van Nguyen1, Tuan Ngo Nguyen1, Bonan Min2, and Thien Huu Nguyen1,3 1 Dept. of Computer and Information Science, University of Oregon, Eugene, OR, USA 2 Raytheon BBN Technologies, USA 3 VinAI Research, Vietnam {minhnv,tnguyen,thien}@cs.uoregon.edu, bonan.min@raytheon.com Abstract Previous work on crosslingual Relation and Event Extraction (REE) suffers from the mono- lingual bias issue due to the training of models on only the source language data. An approach to overcome this issue is to use unlabeled data in the target language to aid the alignment of crosslingual representations, i.e., via fooling a language discriminator. However, as this ap- proach does not condition on class informa- tion, a target language example of a class could be incorrectly aligned to a source language ex- ample of a different class. To address this issue, we propose a novel crosslingual align- ment method that leverages class information of REE tasks for representation learning. In particular, we propose to learn two versions of representation vectors for each class in an REE task based on either source or target lan- guage examples. Representation vectors for corresponding classes will then be aligned to achieve class-aware alignment for crosslingual representations. In addition, we propose to fur- ther align representation vectors for language- universal word categories (i.e., parts of speech and dependency relations). As such, a novel filtering mechanism is presented to facilitate the learning of word category representations from contextualized representations on input texts based on adversarial learning. We con- duct extensive crosslingual experiments with English, Chinese, and Arabic over REE tasks. The results demonstrate the benefits of the pro- posed method that significantly advances the state-of-the-art performance in these settings. 1 Introduction Relation and Event Extraction (REE) are important tasks of Information Extraction (IE), whose goal is to extract structured information from unstructured text (Walker et al., 2006). Due to their complex- ity, annotations for REE tasks are costly and only available in a few languages. Thus, there have been growing interests on crosslingual learning for REE in which a model is trained on a language, i.e., source language, and applied to another lan- guage, i.e., target language, where the annotations are not available. Recent approaches for crosslin- gual REE have mainly employed multilingual word embeddings, e.g., MUSE, (Joulin et al., 2018; Ni and Florian, 2019; Liu et al., 2019; Subburathinam et al., 2019) or multilingual pre-trained language models, e.g., multilingual BERT, (Devlin et al., 2019; M’hamdi et al., 2019; Ahmad et al., 2021; Nguyen and Nguyen, 2021) to learn crosslingual representation vectors for REE. However, previous work on crosslingual REE suffers from the monolingual bias issue due to the monolingual training of models on only the source language data, leading to non-optimal crosslingual performance. A solution for this issue can resort to language adversarial training (Chen et al., 2019; Huang …

Content truncated to 500 words