Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media

Proceedings of NAACL-HLT 2019, pages 2109–2116 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2109 Multi-Task Ordinal Regression for Jointly Predicting the Trustworthiness and the Leading Political Ideology of News Media Ramy Baly1, Georgi Karadzhov2, Abdelrhman Saleh3, James Glass1, Preslav Nakov4 1MIT Computer Science and Artificial Intelligence Laboratory, MA, USA 2SiteGround Hosting EOOD, Bulgaria, 3Harvard University, MA, USA 4Qatar Computing Research Institute, HBKU, Qatar {baly, glass}@mit.edu, georgi.m.karadjov@gmail.com abdelrhman saleh@college.harvard.edu, pnakov@hbku.edu.qa Abstract In the context of fake news, bias, and propa- ganda, we study two important but relatively under-explored problems: (i) trustworthiness estimation (on a 3-point scale) and (ii) po- litical ideology detection (left/right bias on a 7-point scale) of entire news outlets, as op- posed to evaluating individual articles. In par- ticular, we propose a multi-task ordinal re- gression framework that models the two prob- lems jointly. This is motivated by the obser- vation that hyper-partisanship is often linked to low trustworthiness, e.g., appealing to emo- tions rather than sticking to the facts, while center media tend to be generally more impar- tial and trustworthy. We further use several auxiliary tasks, modeling centrality, hyper- partisanship, as well as left-vs.-right bias on a coarse-grained scale. The evaluation results show sizable performance gains by the joint models over models that target the problems in isolation. 1 Introduction Recent years have seen the rise of social media, which has enabled people to virtually share in- formation with a large number of users without regulation or quality control. On the bright side, this has given an opportunity for anyone to be- come a content creator, and has also enabled a much faster information dissemination. However, it has also opened the door for malicious users to spread disinformation and misinformation much faster, enabling them to easily reach audience at a scale that was never possible before. In some cases, this involved building sophisticated profiles for individuals based on a combination of psycho- logical characteristics, meta-data, demographics, and location, and then micro-targeting them with personalized “fake news” with the aim of achiev- ing some political or financial gains (Lazer et al., 2018; Vosoughi et al., 2018). A number of fact-checking initiatives have been launched so far, both manual and automatic, but the whole enterprise remains in a state of cri- sis: by the time a claim is finally fact-checked, it could have reached millions of users, and the harm caused could hardly be undone. An arguably more promising direction is to focus on fact-checking entire news outlets, which can be done in advance. Then, we could fact-check the news before they were even written: by checking how trustworthy the outlets that published them are. Knowing the reliability of a medium is important not only when fact-checking a claim (Popat et al., 2017; Nguyen et al., 2018), but also when solving article-level tasks such as “fake news” and click-bait detection (Brill, 2001; Finberg et al., 2002; Hardalov et al., 2016; Karadzhov et al., 2017; De Sarkar et al., 2018; Pan et al., …

Content truncated to 500 words