We Can Detect Your Bias: Predicting the Political Ideology of News Articles

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 4982–4991, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 4982 We Can Detect Your Bias: Predicting the Political Ideology of News Articles Ramy Baly1, Giovanni Da San Martino2, James Glass1, Preslav Nakov2 1MIT Computer Science and Artificial Intelligence Laboratory 3Qatar Computing Research Institute, HBKU {baly,glass}@mit.edu {gmartino,pnakov}@hbku.edu.qa Abstract We explore the task of predicting the lead- ing political ideology or bias of news articles. First, we collect and release a large dataset of 34,737 articles that were manually annotated for political ideology –left, center, or right–, which is well-balanced across both topics and media. We further use a challenging exper- imental setup where the test examples come from media that were not seen during train- ing, which prevents the model from learning to detect the source of the target news arti- cle instead of predicting its political ideology. From a modeling perspective, we propose an adversarial media adaptation, as well as a spe- cially adapted triplet loss. We further add background information about the source, and we show that it is quite helpful for improv- ing article-level prediction. Our experimental results show very sizable improvements over using state-of-the-art pre-trained Transformers in this challenging setup. 1 Introduction In any piece of news, there is a chance that the viewpoint of its authors and of the media organiza- tion they work for, would be reflected in the way the story is being told. The emergence of the Web and of social media has lead to the proliferation of information sources, whose leading political ide- ology or bias may not be explicit. Yet, systematic exposure to such bias may foster intolerance as well as ideological segregation, and ultimately it could affect voting behavior, depending on the de- gree and the direction of the media bias, and on the voters’ reliance on such media (DellaVigna and Ka- plan, 2007; Iyengar and Hahn, 2009; Saez-Trumper et al., 2013; Graber and Dunaway, 2017). Thus, making the general public aware, e.g., by track- ing and exposing bias in the news is important for a healthy public debate given the important role media play in a democratic society. Media bias can come in many different forms, e.g., by omission, by over-reporting on a topic, by cherry-picking the facts, or by using propaganda techniques such as appealing to emotions, preju- dices, fears, etc. (Da San Martino et al., 2019, 2020a,b) Bias can occur with respect to a spe- cific topic, e.g., COVID-19, immigration, climate change, gun control, etc. (Darwish et al., 2020; Stefanov et al., 2020) It could also be more system- atic, as part of a political ideology, which in the Western political system is typically defined as left vs. center vs. right political leaning. Predicting the bias of individual news articles can be useful in a number of scenarios. For news media, it could be an important element of internal quality assurance as well as of internal or external monitoring for regulatory compliance. For news …

Content truncated to 500 words