Team Jack Ryder at SemEval-2019 Task 4: Using BERT Representations for Detecting Hyperpartisan News

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1012–1015 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1012 Team JACK RYDER at SemEval-2019 Task 4: Using BERT Representations for Detecting Hyperpartisan News Daniel Shaprin1, Giovanni Da San Martino2, Alberto Barr´on-Cede˜no2, Preslav Nakov2 1Sofia University “St Klimen Ohridski”, Sofia, Bulgaria 2Qatar Computing Research Institute, HBKU, Doha, Qatar shaprin@uni-sofia.bg {gmartino, albarron, pnakov}@hbku.edu.qa Abstract We describe the system submitted by the Jack Ryder team to SemEval-2019 Task 4 on Hy- perpartisan News Detection. The task asked participants to predict whether a given article is hyperpartisan, i.e., extreme-left or extreme- right. We propose an approach based on BERT with fine-tuning, which was ranked 7th out 28 teams on the distantly supervised dataset, where all articles from a hyperpartisan/non- hyperpartisan news outlet are considered to be hyperpartisan/non-hyperpartisan. On a manu- ally annotated test dataset, where human an- notators double-checked the labels, we were ranked 29th out of 42 teams. 1 Introduction SemEval-2019 Task 4 (Kiesel et al., 2019) asks to distinguish between articles that are extremely one-sided, i.e., extreme-left or extreme-right, and such that are not. The organizers provided two datasets: 1. By article: A small dataset of 645 manually annotated articles (BA in the following). 2. By publisher: A large dataset of 750,000 articles annotated using distant supervision, where an article is considered hyperpartisan if its source is labeled as such (BP in the fol- lowing). The set is separated into 600,000 articles for training (BP-train) and 150,000 articles for validation (BP-val). Furthermore, two test sets, one annotated by ar- ticle (BA-test) and one annotated by publisher (BP-test), were hidden from the participants and they were used for getting the final scores for the competition. The task is a binary classifica- tion one, where each article is to be assigned one of two possible classes: hyperpartisan and non- hyperpartisan. 2 Related Work Media bias was used as a feature for “fake news” detection (Horne et al., 2018a). It has also been the target of classification, e.g., Horne et al. (2018b) predicted whether an article is biased (political or bias) vs. unbiased. Similarly, Potthast et al. (2018) classified the bias in a target article as (i) left vs. right vs. mainstream, or as (ii) hyper-partisan vs. mainstream. Left-vs-right bias classification at the article level was also explored by Kulkarni et al. (2018), who modeled both text and URL struc- ture. Some work targeted bias at the phrase or the sentence level (Iyyer et al., 2014), for politi- cal speeches (Sim et al., 2013) or legislative docu- ments (Gerrish and Blei, 2011), or targeting users in Twitter (Preot¸iuc-Pietro et al., 2017). More re- cent work has targeted the political bias of entire news outlets (Baly et al., 2018, 2019). Another line of related work focused on propaganda, which is a form of extreme bias (Rashkin et al., 2017; Barr´on-Cede˜no et al., 2019a,b). See also a re- cent position paper (Pitoura et al., 2018) and an overview paper on bias on …

Content truncated to 500 words