Tom Jumbo-Grumbo at SemEval-2019 Task 4: Hyperpartisan News Detection with GloVe vectors and SVM

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1067–1071 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1067 Tom Jumbo-Grumbo at SemEval-2019 Task 4: Hyperpartisan News Detection with GloVe vectors and SVM Chia-Lun Yeh1, Babak Loni2, and Anne Schuth2 1TU Delft, The Netherlands 2De Persgroep c.yeh-1@student.tudelft.nl {babak.loni, anne.schuth}@persgroep.net Abstract In this paper, we describe our attempt to learn bias from news articles. From our experi- ments, it seems that although there is a correla- tion between publisher bias and article bias, it is challenging to learn bias directly from the publisher labels. On the other hand, using few manually-labeled samples can increase the accuracy metric from around 60% to near 80%. Our system is computationally inexpen- sive and uses several standard document repre- sentations in NLP to train an SVM or LR clas- sifier. The system ranked 4th in the SemEval- 2019 task. The code is released for repro- ducibility1. 1 Introduction Bias is the inclination or prejudice for or against one person or group. News articles that con- tain extreme bias fail to provide fair and multi- faceted views for readers and can create polariza- tion within the society (Bernhardt et al., 2008). A system that can detect bias in news articles is thus relevant, especially in a time where an increas- ing number of people consume news from online sources that might not be trustful. The SemEval-2019 task aims to detect hyper- partisan news given the text of the news article, where hyperpartisan news is defined to be an arti- cle that overtly favors a side or view. The details of the task can be found in Kiesel et al. (2019). We are provided with a dataset of two parts. The first part is labeled by the publishers (e.g. if a pub- lisher is decided to be a hyperpartisan source, all its articles are labeled as hyperpartisan), and split into a training and validation set with no overlap- ping publishers (which we will refer to as training- 1 and validation-1). The second part is crowd- sourced and labeled per article (which we will call training-2). 1https://github.com/chialun-yeh/ SemEval2019 Due to the large number of labeled samples, we decide to use a supervised classification approach, where features are extracted from the text and used to train a classifier. Bag-of-words (BoW), TFIDF weighting, and n-grams have been shown to be strong baselines (Hu and Liu, 2004; Wang and Manning, 2012). Other features such as Part-Of- Speech (POS), counts of sentiment and bias words have also been studied (Liu, 2012; Mukherjee and Weikum, 2015). In a similar setting, Potthast et al. (2018) uses features such as n-gram of charac- ters, readability scores, dictionary, and the ratio of quoted words to separate hyperpartisan news from the mainstream. They trained a random for- est classifier and achieved an accuracy of 75%. Kulkarni et al. (2018) build a neural network to predict the political ideology of news articles to be either left, right or center. They combine in- formation …

Content truncated to 500 words