Detecting Promotional Content in Wikipedia

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1851–1857, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Detecting Promotional Content in Wikipedia Shruti Bhosale Heath Vinicombe Raymond J. Mooney Department of Computer Science The University of Texas at Austin {shruti,vini,mooney}@cs.utexas.edu Abstract This paper presents an approach for detecting promotional content in Wikipedia. By incor- porating stylometric features, including fea- tures based on n-gram and PCFG language models, we demonstrate improved accuracy at identifying promotional articles, compared to using only lexical information and meta- features. 1 Introduction Wikipedia is a free, collaboratively edited encyclo- pedia. Since normally anyone can create and edit pages, some articles are written in a promotional tone, violating Wikipedia’s policy requiring a neu- tral viewpoint. Currently, such articles are identified manually and tagged with an appropriate Cleanup message1 by Wikipedia editors. Given the scale and rate of growth of Wikipedia, it is infeasible to man- ually identify all such articles. Hence, we present an approach to automatically detect promotional ar- ticles. Related work in quality flaw detection Wikipedia (Anderka et al., 2012) has relied onin meta-features based on edit history, Wikipedia links, structural features and counts of words, sentences and paragraphs. However, we hypothesize that there are subtle differences in the linguistic style that dis- tinguish promotional tone, which we attempt to cap- ture using stylometric features, particularly deeper syntactic features. We model the style of promo- tional and normal articles using language models 1http://en.wikipedia.org/wiki/Wikipedia: Template_messages/Cleanup based on both n-grams and Probabilistic Context Free Grammars (PCFGs). We show that using such stylometric features improves over using only shal- low lexical and meta-features. 2 Related Work Anderka et al. (2012) developed a general model for detecting ten of Wikipedia’s most frequent quality flaws. One of these flaw types, “Advert”2, refers to articles written like advertisements. Their classifiers were trained using a set of lexical, structural, net- work and edit-history related features of Wikipedia articles. However, they used no features capturing syntactic structure, at a level deeper than Part-Of- Speech (POS) tags. A related area is that of vandalism detection in Wikipedia. Several systems have been developed to detect vandalizing edits in Wikipedia. These fall into two major categories: those analyzing author in- formation and edit metadata (Wilkinson and Huber- man, 2007; Stein and Hess, 2007); and those using NLP techniques such as n-gram language models and PCFGs (Wang and McKeown, 2010; Harpalani et al., 2011). We combine relevant features from both these categories to train a classifier that distin- guishes promotional content from normal Wikipedia articles. 3 Dataset Collection We extracted a set of about 13,000 articles from English Wikipedia’s category, “Category:All arti- 2“Advert” is the flaw-type of majority of the articles in the Category ‘Articles with a promotional tone’. 1851 Content Features Number of characters Number of words Number of sentences Average Word Length Average, Minimum, Maximum Sentence Lengths, Ratio of Maximum to minimum sentence lengths Ratio of long sentences (>48 words) to Short Sen- tences (<33 words) Percentage …

Content truncated to 500 words