Improving Event Detection with Active Learning

Proceedings of Recent Advances in Natural Language Processing, pages 72–77, Hissar, Bulgaria, Sep 7–9 2015. Improving Event Detection with Active Learning Kai Cao† Xiang Li† Miao Fan‡ Ralph Grishman† † New York University, ‡ Tsinghua University {kcao, xiangli, grishman}@cs.nyu.edu fanmiao.cslt.thu@gmail.com Abstract Event Detection (ED), one aspect of In- formation Extraction, involves identifying instances of specified types of events in text. Much of the research on ED has been based on the specifications of the 2005 ACE [Automatic Content Extrac- tion] event task1, and the associated an- notated corpus. However, as the event in- stances in the ACE corpus are not evenly distributed, some frequent expressions in- volving ACE events do not appear in the training data, adversely affecting perfor- mance. In this paper, we demonstrate the effectiveness of a Pattern Expansion tech- nique to import frequent patterns extracted from external corpora to boost ED per- formance. The experimental results show that our pattern-based system with the ex- panded patterns can achieve 70.4% (with 1.6% absolute improvement) F-measure over the baseline, an advance over current state-of-the-art systems. 1 Introduction Event Extraction involves the extraction of partic- ular types of events along with their arguments. In this paper we shall focus on a subproblem, that of Event Detection (ED) – identifying instances of specified types of events in text. In keeping with the design of the ACE [Automatic Content Extrac- tion] Event task, we will associate each event men- tion with a trigger, which is a word or a sequence of words (most often a single verb or nominaliza- tion) that expresses that event. More precisely, our task involves identifying event triggers and classi- fying them into specific types. For instance, ac- 1http://www.itl.nist.gov/iad/mig/ tests/ace/ cording to the ACE 2005 annotation guidelines2, in the sentence “She was killed in an automo- bile accident yesterday”, an event detection sys- tem should be able to recognize the word “killed” as a trigger for the event DIE. This task is quite challenging, as the same event might appear in the form of various trigger expressions and an expres- sion might represent different events in different contexts. ED is a crucial component in the overall Event Extraction task, which also requires event argument identification and argument role label- ing. Most recent research work on the ACE Event Detection task relies on pattern-based or feature- based approaches, creating classifiers for trigger labeling. Since the distribution of ACE event types in the corpus is skewed, the test data in- cludes some relatively common event expressions that do not occur in the training data. To over- come this problem, we propose to use active learn- ing to help include more patterns for boosting ED performance. These patterns will be extracted from external corpora, such as the EnglishGiga- Word corpus, labeled, and added to the training data. The experimental results demonstrate that our pattern-based system with the expanded pat- terns can achieve 70.4% (with 1.6% absolute im- provement) F-measure over the baseline, an ad- vance over the state-of-the-art systems. The paper …

Content truncated to 500 words