NwQM: A neural quality assessment framework for Wikipedia

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 8396–8406, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 8396 NwQM: A neural quality assessment framework for Wikipedia Bhanu Prakash Reddy∗1,Sasi Bhushan*2,Soumya Sarkar*3,Animesh Mukherjee4 IIT Kharagpur, India2,3,4, Adobe Research, India1 soumya015@iitkgp.ac.in3,guda@adobe.com1 sasibhushan3@gmail.com2, animesh@cse.iitkgp.ac.in4 Abstract Millions of people irrespective of socio- economic and demographic backgrounds, de- pend on Wikipedia articles everyday for keep- ing themselves informed regarding popular as well as obscure topics. Articles have been categorized by editors into several qual- ity classes, which indicate their reliability as encyclopedic content. This manual designa- tion is an onerous task because it necessitates profound knowledge about encyclopedic lan- guage, as well navigating circuitous set of wiki guidelines. In this paper we propose Neural wikipedia Quality Monitor (NwQM), a novel deep learning model which accumulates signals from several key information sources such as article text, meta data and images to obtain improved Wikipedia article representa- tion. We present comparison of our approach against a plethora of available solutions and show 8% improvement over state-of-the-art ap- proaches with detailed ablation studies. 1 Introduction Wikipedia is one of the most prominent sources of free information in the world today. Since reli- able and advertisement free material is mostly be- hind pay-walled sources, a huge volume of global population is directed toward this extraordinary crowd sourced platform for information ranging from history, politics, pop-culture to even scientific topics (Horta Ribeiro et al., 2020). Although Wikipedia has grown significantly in terms of volume and veracity over the last decade, the quality of articles is not uniform (Warncke- Wang et al., 2015). The quality of Wikipedia ar- ticles is monitored through a rating system where each article is assigned one of several class indi- cators. Some of the major article categories are FA, GA, B, C, Start and Stub. Most complete ∗*Authors contributed equally and dependable content is annotated by an FA (aka featured article) tag while lowest quality content is annotated with a Stub tag. The intention behind this elaborate scheme is to notify editors regard- ing current state of the article and extent of effort needed for escalating to encyclopedic standards1. There exist several guidelines which direct ed- itors in annotating articles into respective classes. Some of the traits of a FA article are engaging and comprehensive prose with neutral point of view and verifiable claims. It must also rigorously fol- low the style manual, i.e. the page structure. Fur- ther the content should be stable, i.e. devoid of edit warring. Understanding compliance with these guidelines often require detailed knowledge about language usage as well as domain knowledge about Wikipedia page layout and style principles. Often it is nontrivial to discern qualifying differences be- tween articles which merits their ratings without inculcating personal biases. Consider the wikipage of two prominent US presidents Abraham Lincoln (GA) and John F. Kennedy (B). Both pages are indistinguishable in-terms of coverage and engagement, however on closer assessment it is apparent that President Kennedy’s page contains unattributed opinion …

Content truncated to 500 words