Automatically Classifying Edit Categories in Wikipedia Revisions

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 578–589, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Automatically Classifying Edit Categories in Wikipedia Revisions Johannes Daxenberger† and Iryna Gurevych†‡ † Ubiquitous Knowledge Processing Lab Department of Computer Science, Technische Universit¨at Darmstadt ‡ Information Center for Education German Institute for Educational Research and Educational Information http://www.ukp.tu-darmstadt.de Abstract In this paper, we analyze a novel set of fea- tures for the task of automatic edit category classification. Edit category classification as- signs categories such as spelling error correc- tion, paraphrase or vandalism to edits in a doc- ument. Our features are based on differences between two versions of a document includ- ing meta data, textual and language properties and markup. In a supervised machine learning experiment, we achieve a micro-averaged F1 score of .62 on a corpus of edits from the En- glish Wikipedia. In this corpus, each edit has been multi-labeled according to a 21-category taxonomy. A model trained on the same data achieves state-of-the-art performance on the related task of fluency edit classification. We apply pattern mining to automatically la- beled edits in the revision histories of different Wikipedia articles. Our results suggest that high-quality articles show a higher degree of homogeneity with respect to their collabora- tion patterns as compared to random articles. 1 Introduction Due to its ever-evolving and collaboratively built content, Wikipedia has been the subject of many NLP studies. While the number of newly created articles in the online encyclopedia declined in the last few years (Suh et al., 2009), the number of edits in existing articles is rather stable.1 It is reasonable to assume that the latter will not change in the near 1http://stats.wikimedia.org/EN/ TablesDatabaseEdits.htm future. One of the major reasons for the popular- ity of Wikipedia is its up-to-dateness (Keegan et al., 2013), which in turn requires constant editing activ- ity. Wikipedia’s revision history stores all changes made to any page in the encyclopedia in separate revisions. Previous studies have exploited revision history data in tasks such as preposition error cor- rection (Cahill et al., 2013), spelling error correc- tion (Zesch, 2012) or paraphrasing (Max and Wis- niewski, 2010). However, they all use different ap- proaches to extract the information needed for their task. Ferschke et al. (2013) outline several appli- cations benefiting from revision history data. They argue for a unified approach to extract and classify edits from revision histories based on a predefined edit category taxonomy. In this work, we show how the extraction and automatic multi-label classification of any edit in Wikipedia can be handled with a single approach. Therefore, we use the 21-category edit classification taxonomy developed in previous work (Daxenberger and Gurevych, 2012). This taxonomy enables a fine- grained analysis of edit activity in revision histories. We present the results from an automatic classifica- tion experiment, based on an annotated corpus of ed- its in the English Wikipedia. Additional information necessary to reproduce our results, including word lists and training, development …

Content truncated to 500 words