Unsupervised Domain Adaptation for Event Detection using Domain-specific Adapters

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4015–4025 August 1–6, 2021. ©2021 Association for Computational Linguistics 4015 Unsupervised Domain Adaptation for Event Detection using Domain-specific Adapters Nghia Ngo Trung1, Duy Phung1 and Thien Huu Nguyen2 1 VinAI Research, Vietnam 2 Department of Computer and Information Science, University of Oregon, Eugene, OR 97403, USA {v.nghiant66,v.duypv1}@vinai.io, thien@cs.uoregon.edu Abstract Due to the multi-dimensional variation of tex- tual data, detection of event triggers from new domains can become a lot more challenging. This prompts a need to research on domain adaptation methods for event detection task, especially for the most practical unsupervised setting. Recently, large transformer-based lan- guage models, e.g. BERT, have become essen- tial to achieve top performance for event detec- tion. However, their unwieldy nature also pre- vents effective adaptation across domains. To this end, this work proposes a Domain-specific Adapter-based Adaptation (DAA) framework to improve the adaptability of BERT-based models for event detection across domains. By explicitly representing data from differ- ent domains with separate adapter modules in each layer of BERT, DAA introduces a novel joint representation learning mechanism and a Wasserstein distance-based technique for data selection in adversarial learning to substan- tially boost the performance on target domains. Extensive experiments and analysis over dif- ferent datasets (i.e., LitBank, TimeBank, and ACE-05) demonstrate the effectiveness of our approach. 1 Introduction Event detection (ED) is an important component in the overall event extraction pipeline, which plays a crucial role in any natural language understand- ing system. The goal of ED is to identify event triggers in a given text and classify them into one of several pre-defined types. Formally, according to the ACE-05 annotation guideline , each event trigger is a phrase (usually a single verb or nomi- nalization), which evokes that event in the context of the associating event mention. For example, the word “fired” is the trigger word for an event of type Attack in the following sentence: ”The police fired tear gas and water cannons in street battles with activists.” Tackling ED problem involves both locating the event triggers and categorizing them into specific event types, therefore can be a quite challenging task due to the intricate dependency among triggers, events, and contexts in linguistic data. The complication is further amplified by do- main shift problem when text are collected from multiple different domains. The majority of prior approaches on ED relied on the basic supervised learning assumption where training and testing data follow the same distribu- tion. Several works further evaluated their methods on cross-domain setting where their models were trained using data from one domain and tested on another, without leveraging any adaptation mecha- nism to alleviate the domain shift problem (Nguyen and Grishman, 2015; Yubo et al., 2015; Hong et al., 2018b). To this end, our work explores the gen- eral problem of domain adaptation for ED where data comes from two different source and target do- mains. In particular, we focus on the unsupervised setting that requires no annotations for target …

Content truncated to 500 words