Using Prediction from Sentential Scope to Build a Pseudo Co-Testing Learner for Event Extraction

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 714–722, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Using Prediction from Sentential Scope to Build a Pseudo Co-Testing Learner for Event Extraction Shasha Liao Ralph Grishman Computer Science Department Computer Science Department New York University New York University liaoss@cs.nyu.edu grishman@cs.nyu.edu Abstract Event extraction involves the identifica- tion of instances of a type of event, along with their attributes and participants. De- veloping a training corpus by annotating events in text is very labor intensive, and so selecting informative instances to an- notate can save a great deal of manual work. We present an active learning (AL) strategy, pseudo co-testing, based on one view from a classifier aiming to solve the original problem of event ex- traction, and another view from a classifi- er aiming to solve a coarser granularity task. As the second classifier can provide more graded matching from a wider scope, we can build a set of pseudo- contention-points which are very in- formative, and can speed up the AL pro- cess. Moreover, we incorporate multiple selection criteria into the pseudo co- testing, seeking training examples that are informative, representative, and varied. Experiments show that pseudo co-testing can reduce annotation labor by 81%; in- corporating multiple selection criteria re- duces the labor by a further 7%. 1 Introduction The goal of event extraction is to identify instances of a class of events in text. There can be many event types; for example, the ACE 2005 event extraction task involved a set of 33 generic event types and subtypes appearing frequently in the news. A typical event extraction task, in addition to identifying the event itself, also identifies all of the participants and attributes of the event; these are the entities that are involved in that event. Annotating a corpus in order to train an event tagger is a costly task. First of all, event extraction is difficult and requires substantial training data. The same event might be presented in various expressions, and an expression might represent different events in different contexts. For example, “retire” and “resign” can both represent an End-Position event, while “leave” can represent either an End- Position or Move event in different contexts. Moreover, for each event type, the event participants and attributes may also appear in multiple forms and exemplars of the different forms may be required. Furthermore, compared to other tasks like name tagging or part of speech tagging, events of a particular type appear relatively rarely in a document. One document might only contain one or two events of a given type, or even none at all. For the ACE 2005 event extraction task, Attack events have the highest frequency in the training corpus (2240 times, an average of 4 events per document), while Start-Position events only appear 232 times (an average of 1/3 event per document). As a result, to acquire enough training samples, we need to annotate a lot of documents. If we can predict …

Content truncated to 500 words