Modeling Document-Level Context for Event Detection via Important Context Selection

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5403–5413 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5403 Modeling Document-Level Context for Event Detection via Important Context Selection Amir Pouran Ben Veyseh1, Minh Van Nguyen1, Nghia Ngo Trung2, Bonan Min3, and Thien Huu Nguyen1 1 Dept. of Computer and Information Science, University of Oregon, Eugene, OR, USA 2 VinAI Research, Vietnam 3 Raytheon BBN Technologies, USA {apouranb,minhnv,thien}@cs.uoregon.edu, v.nghiant66@vinai.io,bonan.min@raytheon.com Abstract Event Detection (ED) aims to recognize and classify trigger words of events in text. The recent progress has featured ad- vanced transformer-based language models (e.g., BERT) as a critical component in state- of-the-art models for ED. However, the length limit for input texts is a barrier for such ED models as they cannot encode long-range document-level context that has been shown to be beneficial for ED. To address this issue, we propose a novel method to model document- level context with BERT for ED that dynam- ically selects relevant sentences in the docu- ment for the event prediction of the target sen- tence. The target sentence will be then aug- mented with the selected sentences and con- sumed entirely by BERT for improved repre- sentation learning for ED. To this end, the RE- INFORCE algorithm is employed to train the relevant sentence selection for ED. Several in- formation types are then introduced to form the reward function for the training process, in- cluding ED performance, sentence similarity, and discourse relations. Our extensive exper- iments on multiple benchmark datasets reveal the effectiveness of the proposed model, lead- ing to new state-of-the-art performance. 1 Introduction Event Detection (ED) is one of the fundamental tasks for Information Extraction. Its goal is to identify the word(s) in text that most clearly evoke events and classifying them into predefined event types (event trigger words). For instance, in the input sentence “After the scandal, David James was fired from the company.", an ED system needs to recognize the word “fired” as an event trigger and predict its event type as End-Position. The early methods for ED have involved feature- based models (Ahn, 2006; Liao and Grishman, 2010a; Miwa et al., 2014) while recent work has featured deep learning methods (Nguyen and Gr- ishman, 2015; Chen et al., 2015; Lin et al., 2020; Nguyen et al., 2021). Similar to other NLP tasks, the current best systems for ED leverage transformer-based language models, e.g., BERT (Devlin et al., 2019), as a critical encoding compo- nent to achieve state-of-the-art performance (Lai et al., 2020b; Lin et al., 2020). As such, most of the current transformer-based models for ED only focus on sentence-level context in which the scope of context to predict event type for each word is limited to the host sentence (Lu et al., 2019; Wang et al., 2019). However, it has been shown that document-level context also provides important in- formation for deep learning models for ED (Chen et al., 2018; Zhao et al., 2018). For instance, in the document “The troops were …

Content truncated to 500 words