Expectation-Regulated Neural Model for Event Mention Extraction

Proceedings of NAACL-HLT 2016, pages 400–410, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Expectation-Regulated Neural Model for Event Mention Extraction Ching-Yun Chang and Zhiyang Teng and Yue Zhang Singapore University of Technology and Design 8 Somapah Road, Singapore 487372 {chingyun chang, yue zhang}@sutd.edu.sg zhiyang teng@mymail.sutd.edu.sg Abstract We tackle the task of extracting tweets that mention a specific event from all tweets that contain relevant keywords, for which the main challenges include unbalanced positive and negative cases, and the unavailability of man- ually labeled training data. Existing meth- ods leverage a few manually given seed events and large unlabeled tweets to train a classi- fier, by using expectation regularization train- ing with discrete ngram features. We pro- pose a LSTM-based neural model that learns tweet-level features automatically. Compared with discrete ngram features, the neural model can potentially capture non-local dependen- cies and deep semantic information, which are more effective for disambiguating subtle se- mantic differences between true event men- tions and false cases that use similar word- ing patterns. Results on both tweets and fo- rum posts show that our neural model is more effective compared with a state-of-the-art dis- crete baseline. 1 Introduction A Distributed Denial of Service (DDoS) attack em- ploys multiple compromised systems to interrupt or suspend services of a host connected to the Internet. Victims are often high-profile web servers such as banks or credit card payment gateways, and there- fore a single attack may cause considerable loss. The aim of this paper is to build an automatic system which can extract DDoS event mentions from social media, a timely information source for events taking place around the world, so that the mined emerging incidents can serve as early DDoS warnings or signs for Internet service providers. Ritter et al. (2015) proposed the first work to ex- tract cybersecurity event mentions from raw Twitter stream. They investigated three different event cat- egories, namely DDoS attacks, data breaches and account hijacking, by tracking the keywords ddos, breach and hacked, respectively. Not all tweets con- taining the keywords describe events. For example, the tweet “give me paypall or i will tell my mum and ddos u” shows a metaphor rather than a DDoS event. As a result, the event mention extraction task involves a classification task that filters out true events from all tweets that contain event keywords. Two main challenges exist for this task. First, the numbers of positive and negative examples are typ- ically unbalanced. In our datasets, only about 22% of the tweets that contain the term ddos are men- tions to DDoS attack events. Second, there is typ- ically little manual annotation available. Ritter et al. (2015) tackled the challenges by weakly super- vising a classification model with a small number of human-provided seed events. In particular, Ritter et al. exploit expectation regularization (ER; Mann and McCallum (2007)) for semi-supervised learning from large amounts of raw tweets that contain the event keyword. They show that the ER approach outperforms semi- supervised …

Content truncated to 500 words