Investigating Statistical Techniques for Sentence-Level Event Classification

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 617–624 Manchester, August 2008 Investigating Statistical Techniques for Sentence-Level Event Classification Martina Naughton School of Computer Science, University College Dublin, Belfield, Dublin 4, Ireland Nicola Stokes NICTA Victoria Laboratory, University of Melbourne, Victoria, Australia Joe Carthy School of Computer Science University College Dublin Belfield, Dublin 4, Ireland Abstract The ability to correctly classify sentences that describe events is an important task for many natural language applications such as Question Answering (QA) and Sum- marisation. In this paper, we treat event detection as a sentence level text classifi- cation problem. We compare the perfor- mance of two approaches to this task: a Support Vector Machine (SVM) classifier and a Language Modeling (LM) approach. We also investigate a rule based method that uses hand crafted lists of terms derived from WordNet. These terms are strongly associated with a given event type, and can be used to identify sentences describing in- stances of that type. We use two datasets in our experiments, and evaluate each tech- nique on six distinct event types. Our re- sults indicate that the SVM consistently outperform the LM technique for this task. More interestingly, we discover that the manual rule based classification system is a very powerful baseline that outperforms the SVM on three of the six event types. 1 Introduction Event detection is a core Natural Language Pro- cessing (NLP) task that focuses on the automatic identification and classification of various event types in text. This task has applications in au- tomatic Text Summararisation and Question An- swering (QA). For example, event recognition is a core task in QA since the majority of web user c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. questions have been found to relate to events and situations in the world (Saur´ı et al., 2005). For complex questions such as How many people were killed in Baghdad in March?, QA systems often rely on event detection systems to identify all rel- evant events in a set of documents before formu- lating an answer. More recently, much research in summarisation has focused on the use of phrasal concepts such as events to represent sentences in extractive summarisation systems. Specifically, (Filatova and Hatzivassiloglou, 2004) use event- based features to represent sentences and shows that this approach improves the quality of the final summaries when compared with a baseline bag-of- words approach. In this paper, we investigate the use of statistical methods for identifying the sentences in a docu- ment that describe one or more instances of a spec- ified event type. We treat this task as a text classi- fication problem where each sentence in a given document is either classified as containing an in- stance of the target event or not. We view this task as a filtering step in a larger pipeline NLP ar- chitecture (e.g. a QA system) which helps speed up subsequent processing by removing irrelevant, non-event sentences. Two …

Content truncated to 500 words