Modeling Newswire Events using Neural Networks for Anomaly Detection

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1414–1422, Dublin, Ireland, August 23-29 2014. Modeling Newswire Events using Neural Networks for Anomaly Detection Pradeep Dasigi Language Technologies Institute 5000 Forbes Avenue Pittsburgh, PA 15213 USA pdasigi@cs.cmu.edu Eduard Hovy Language Technologies Institute 5000 Forbes Avenue Pittsburgh, PA 15213 USA hovy@cmu.edu Abstract Automatically identifying anomalous newswire events is a hard problem. We discuss the com- plexity of the problem and introduce a novel technique to model events based on recursive neural networks to represent events as composition of their semantic arguments. Our model learns to differentiate between normal and anomalous events. We model anomaly detection as a binary classification problem and show that the model learns useful features to classify anomaly. We use headlines from the weird news category publicly available on newswire websites to extract anomalous training examples and those from Gigaword as normal examples. We evaluate the classifier on human annotated data and obtain an accuracy of 65.44%. We also show that our model is at least as competent as the least competent human annotator in anomaly detection. 1 Introduction Understanding events is a fundamental prerequisite for deeper semantic analysis of language. We intro- duce the problem of automatic anomalous event detection in this paper and propose a novel event model that can learn to differentiate between normal and anomalous events. We generally define anomalous events as those that are unusual compared to the general state of affairs and might invoke surprise when reported. For example, given the event mention in the following sentence Man recovering after being shot by his dog. one might think it is strange because dogs are not expected to shoot men. But the mentions Man recovering after being shot by cops. Man recovering after being bitten by a dog. are not as unusual as the previous one. While all three sentences are equally valid syntactically, and it is not unclear what any of them means, it is our knowledge about the role fillers —both individually and specifically in combination— that enables us to differentiate between normal and anomalous events. Hence we hypothesize that anomaly is a result of unexpected or unusual combination of semantic role fillers. Given this idea, an automatic anomaly detection algorithm has to encode the goodness of semantic role filler coherence. It has to be noted that event level anomaly is not the same as semantic incoherence. An event con- structed by randomly choosing words to form each of the semantic arguments is not anomalous since we cannot argue whether the event is normal or anomalous when it is unclear what the event means. Hence, we define anomalous events to be the sub class of those that are semantically coherent, but are unusual only based on real world knowledge. Automatic anomalous event detection is a hard problem since determining what a good combination of role fillers requires deep semantic and pragmatic knowledge. Moreover, manual judgment of anomaly This work is licenced under a Creative Commons Attribution …

Content truncated to 500 words