Identifying Event Descriptions using Co-training with Online News Summaries

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 282–291, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Identifying Event Descriptions using Co-training with Online News Summaries William Yang Wang and Kapil Thadani and Kathleen R. McKeown Computer Science Department Columbia University yww@cs.cmu.edu, {kapil, kathy}@cs.columbia.edu Abstract Systems that distill information about events from large corpora generally extract sentences that are relevant to a short event query. We present a novel co-training strategy for this task that employs a multi- document news summary corpus featuring 2.5 million unlabeled sentences, thus obvi- ating the need for extensive manual anno- tation. Our experiments indicate that this technique significantly outperforms stan- dard classification approaches with linear feature combination on this task. An anal- ysis of our approach under various set- tings reveals how classifier and parameter choice can be used to control runtime over- head while contributing to an absolute in- crease of 22% in recall. 1 Introduction Automatic identification of event descriptions is a crucial, yet difficult, problem with impact on applications such as question answering (QA), query-focused summarization (QS), and text min- ing (TM) systems. In such applications, a system is given a description of an event in the form of a query and the task is to identify sentences within relevant documents (retrieved by an information retrieval system) that describe the event. We de- fine event-relevant sentences as those describing a unique event as specified in the query, typically occurring at a specific location, on a specific date, or with specific participants. The task is made dif- ficult by the fact that words in the query rarely provide enough identifying information (Xu and Croft, 2000; Chirita et al., 2007) to reliably find descriptive sentences; the query may not specify all named entities and may provide other descrip- tive terms that are not easily matched. For ex- ample, given the query requesting a description of “the Israeli-Palestinian peace talks”, the sentence “The discussions between Israelis and Palestini- ans follow last month’s Annapolis meeting where Israeli Prime Minister and Palestinian President Mahmoud Abbas met and agreed to try to nego- tiate a deal before the end of 2008.” is event- relevant, whereas the sentence “Turkey has close ties to both Israel and the Palestinians.” is not. Yet both sentences feature the named entities from the query. The term “peace talks” does not appear in the event-relevant sentence, but is implied by “negotiate”. Previous approaches have addressed this prob- lem using supervised learning (Mani et al., 2003; Bethard and Martin, 2006; Manshadi et al., 2008), where the relation between the query and event- relevant sentences is learned. Still, gathering a large amount of training data is difficult and peo- ple often do not agree on what counts as rele- vant to an event description (Filatova and Hatzi- vassiloglou, 2003), making the process of manu- ally labeling sentences as events both time con- suming and expensive. This is supported by our own experiments with Amazon Mechanical Turk (AMT); we were …

Content truncated to 500 words