A Hybrid Model for Quality Assessment of Wikipedia Articles

A Hybrid Model for Quality Assessment of Wikipedia Articles Aili Shen Jianzhong Qi Timothy Baldwin School of Computing and Information Systems The University of Melbourne Victoria, Australia ailis@student.unimelb.edu.au jianzhong.qi@unimelb.edu.au tb@ldwin.net Abstract The task of document quality assessment is a highly complex one, which draws on analysis of aspects including linguistic content, document structure, fact correct- ness, and community norms. We explore the task in the context of a Wikipedia ar- ticle assessment task, and propose a hy- brid approach combining deep learning with features proposed in the literature. Our method achieves 6.5% higher accu- racy than the state of the art in predicting the quality classes of English Wikipedia articles over a novel dataset of around 60k Wikipedia articles. We also discuss limita- tions with this task setup, and possible di- rections for establishing more robust doc- ument quality assessment evaluations. 1 Introduction With the advent of Web 2.0, it has become much easier to collaboratively write and distribute docu- ments, such as Wikipedia articles, or questions and answers in StackOverflow. The quality of such documents, however, varies greatly, ranging from well-written documents to poorly written docu- ments with little if any content. Automatic quality assessment is needed for the following reasons. First, the number of digital documents being generated is huge and continues to grow. It is infeasible to assess the quality of all documents manually and in a timely manner. Sec- ond, even given the same grading scheme, people assess quality of documents in a subjective way, which makes it difficult to reach consensus among raters and assign a proper quality class to a docu- ment. An automatic labeling approach, however, can enable immediate feedback to both contrib- utors and readers, ideally with a justification for why a given label has been assigned. A high qual- ity class assignment can give users greater trust in the document, while a low-quality class assign- ment can direct the efforts of contributors to im- prove certain articles. In the absence of a general-purpose document quality assessment dataset, we use and expand on a document quality dataset sourced from English Wikipedia. The quality of Wikipedia articles is in- consistent, for reasons including: (1) not all con- tributors (i.e., users who edit a Wikipedia article) are experts in the area of the articles they edit, and different contributors have different writing styles; (2) some articles receive more attention than oth- ers, resulting in imbalances in the level of peer; (3) there is article vandalism (Wikipedia Vandalism) that lowers quality of Wikipedia articles. Officially, there are six quality classes of Wikipedia articles, which are (in descending or- der of quality): Featured Article (FA), Good Ar- ticle (GA), B-class Article (B), C-class Article (C), Start Article (Start), and Stub Article (Stub). The quality class of an article is determined by Wikipedia reviewers, and any registered user can become a reviewer. A general description of the criteria of different quality classes can be found in the Wikipedia grading scheme page.1 The differ- ence between different quality …

Content truncated to 500 words