A comparison and analysis of models for event trigger detection

A comparison and analysis of models for event trigger detection Shang Chun Sam Wei School of Information Technologies University of Sydney NSW 2006, Australia swei4829@uni.sydney.edu.au Ben Hachey School of Information Technologies University of Sydney NSW 2006, Australia ben.hachey@gmail.com Abstract Interpreting event mentions in text is cen- tral to many tasks from scientific research to intelligence gathering. We present an event trigger detection system and explore baseline configurations. Specifically, we test whether it is better to use a single multi-class classifier or separate binary classifiers for each label. The results sug- gest that binary SVM classifiers outper- form multi-class maximum entropy by 6.4 points F-score. Brown cluster and Word- Net features are complementary with more improvement from WordNet features. 1 Introduction Events are frequently discussed in text, e.g., crim- inal activities such as violent attacks reported in police reports, corporate activities such as merg- ers reported in business news, biological processes such as protein interactions reported in scientific research. Interpreting these mentions is central to tasks like intelligence gathering and scientific re- search. Event extraction automatically identifies the triggers and arguments that constitute a textual mention of an event in the world. Consider: Bob bought the book from Alice. Here, a trigger – “bought” (Transaction.Transfer– Ownership) – predicates an interaction between the arguments – “Bob” (Recipient), “the book” (Thing) and “Alice” (Giver). We focus on the trig- ger detection task, which is the first step in event detection and integration. Many event extraction systems use a pipelined approach, comprising a binary classifier to detect event triggers followed by a separate multi-class classifier to label the type of event (Ahn, 2006). Our work is different in that we use a single clas- sification step with sub-sampling to handle data skew. Chen and Ji (2009) use Maximum Entropy (ME) classifier in their work. However, their ap- proach is similar to (Ahn, 2006) where they iden- tify the trigger first then classify the trigger at later stage. Kolya et al. (2011) employ a hybrid ap- proach by using Support Vector Machine (SVM) classifier and heuristics for event extraction. We present an event trigger detection system that formulates the problem as a token-level clas- sification task. Features include lexical and syn- tactic information from the current token and sur- rounding context. Features also include addi- tional word class information from Brown clus- ters, WordNet and Nomlex to help generalise from a fairly small training set. Experiments explore whether multi-class or binary classification is bet- ter using SVM and ME. Contributions include: (1) A comparison of bi- nary and multi-class versions of SVM and ME on the trigger detection task. Experimental results suggest binary SVM outperform other approaches. (2) Analysis showing that Brown cluster, Nom- lex and WordNet features contribute nearly 10 points F-score; WordNet+Nomlex features con- tribute more than Brown cluster features; and im- provements from these sources of word class in- formation increase recall substantially, sometimes at the cost of precision. 2 Event Trigger Detection Task We investigate the event trigger detection task from …

Content truncated to 500 words