“Got You!”: Automatic Vandalism Detection in Wikipedia with Web-based Shallow Syntactic-Semantic Modeling

Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1146–1154, Beijing, August 2010 “Got You!”: Automatic Vandalism Detection in Wikipedia with Web-based Shallow Syntactic-Semantic Modeling William Yang Wang and Kathleen R. McKeown Department of Computer Science Columbia University yw2347@columbia.edu kathy@cs.columbia.edu Abstract Discriminating vandalism edits from non-vandalism edits in Wikipedia is a challenging task, as ill-intentioned edits can include a variety of content and be expressed in many different forms and styles. Previous studies are limited to rule-based methods and learning based on lexical features, lacking in linguistic analysis. In this paper, we propose a novel Web-based shallow syntactic- semantic modeling method, which utiliz- es Web search results as resource and trains topic-specific n-tag and syntactic n-gram language models to detect van- dalism. By combining basic task-specific and lexical features, we have achieved high F-measures using logistic boosting and logistic model trees classifiers, sur- passing the results reported by major Wikipedia vandalism detection systems. 1 Introduction Online open collaboration systems are becoming a major means of information sharing on the Web. With millions of articles from millions of resources edited by millions of people, Wikipe- dia is a pioneer in the fast growing, online know- ledge collaboration era. Anyone who has Inter- net access can visit, edit and delete Wikipedia articles without authentication. A primary threat to this convenience, however, is vandalism, which has become one of Wikipe- dia’s biggest concerns (Geiger, 2010). To date, automatic countermeasures mainly involve rule- based approaches and these are not very effec- tive. Therefore, Wikipedia volunteers have to spend a large amount of time identifying vanda- lized articles manually, rather than spending time contributing content to the articles. Hence, there is a need for more effective approaches to automatic vandalism detection. In contrast to spam detection tasks, where a full spam message, which is typically 4K Bytes (Rigoutsos and Huynh, 2004), can be sampled and analyzed (Itakura and Clarke, 2009), Wiki- pedia vandals typically change only a small number of words or sentences in the targeted article. In our preliminary corpus (Potthast et al., 2007), we find the average size of 201 vanda- lized texts to be only 1K Byte. This leaves very few clues for vandalism modeling. The question we address in this paper is: given such limited information, how can we better understand and model Wikipedia vandalism? Our proposed approach establishes a novel classification framework, aiming at capturing vandalism through an emphasis on shallow syn- tactic and semantic modeling. In contrast to pre- vious work, we recognize the significance of natural language modeling techniques for Wiki- pedia vandalism detection and utilize Web search results to construct our shallow syntactic and semantic models. We first construct a base- line model that captures task-specific clues and lexical features that have been used in earlier work (Potthast et al., 2008; Smets et al., 2008) augmenting these with shallow syntactic and semantic features. Our main contributions are:  Improvement over previous modeling me- thods with three novel lexical features  Using Web search results …

Content truncated to 500 words