Linguistic Benchmarks of Online News Article Quality

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1893–1902, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Linguistic Benchmarks of Online News Article Quality Ioannis Arapakis∗ Eurecat Barcelona, Spain arapakis.ioannis@gmail.com Filipa Peleja∗ Vodafone Lisbon, Portugal filipapeleja@gmail.com B. Barla Cambazoglu∗ Independent Researcher barla@berkantbarlacambazoglu.com Joao Magalhaes NOVA-LINCS, DI, FCT Universidade NOVA Lisboa, Portugal jmag@fct.unl.pt Abstract Online news editors ask themselves the same question many times: what is miss- ing in this news article to go online? This is not an easy question to be answered by computational linguistic methods. In this work, we address this important question and characterise the constituents of news article editorial quality. More specifically, we identify 14 aspects related to the con- tent of news articles. Through a correla- tion analysis, we quantify their indepen- dence and relation to assessing an article’s editorial quality. We also demonstrate that the identified aspects, when combined to- gether, can be used effectively in quality control methods for online news. 1 Introduction A recent study1 found that online news is nowa- days the main source of news for the population in the 18-29 age group (71%), and as popular as TV in the 30-39 age group (63%). The readers appetite for high-quality online news result in an offer of thousands of articles published every day in the whole of the Web. For instance, it is not un- common to find the same facts reported by many different online news articles. However, only a few of them actually grab the attention of the read- ers. Journalists and editors follow standardised discourse rules and techniques aiming at engaging the reader in the article’s narrative of article (Louis and Nenkova, 2013). Analysing the discourse of such articles is cen- tral to properly assessing the quality of online ∗This work was done while the authors were at Ya- hoo!Research Barcelona. 1http://www.people-press.org/2013/08/08/amid- criticism-support-for-medias-watchdog-role-stands-out news (van Dijk and Kintsch, 1983). Defining the variables that computational linguistics should quantify is a challenging task. Several questions arise from this exercise. For example, what does the quality refer to? What makes a new article per- ceived as high quality by the editors/users? What aspects of an article correlate better with its per- ceived quality? Can we predict the quality of an article using linguistic features extracted from its content? These are the kind of questions we ad- dress in this paper. To this end, we propose a linguistic resource and assessment methodology to quantify the ed- itorial quality of online news discourse. We ar- gue that quality is too complex to be represented by a single number and should be instead decom- posed into a set of simpler variables that capture the different linguistic and narrative aspects of on- line news. Thus, we depart from current literature and propose a multidimensional representation of quality. The first contribution of this paper is a taxonomy of 14 different content aspects that are associated with the editor-perceived quality of on- line news articles. The proposed 14 aspects …

Content truncated to 500 words