Team Kermit-the-frog at SemEval-2019 Task 4: Bias Detection Through Sentiment Analysis and Simple Linguistic Features

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1016–1020 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1016 Team Kermit-the-frog at SemEval-2019 Task 4: Bias Detection Through Sentiment Analysis and Simple Linguistic Features Talita Anthonio University of Groningen University of the Basque Country t.r.anthonio@student.rug.nl Lennart Kloppenburg lennartkloppenburg@live.nl Abstract In this paper we describe our participation in the SemEval 2019 shared task on hyperpar- tisan news detection. We present the sys- tem that we submitted for final evaluation and the three approaches that we used: sentiment, bias-laden words and filtered n-gram features. Our submitted model is a Linear SVM that solely relies on the negative sentiment of a document. We achieved an accuracy of 0.621 and a f1 score of 0.694 in the competition, re- vealing the predictive power of negative sen- timent for this task. There was no major im- provement by adding or substituting the fea- tures of the other two approaches that we tried. 1 Introduction With the growing role of social media in politics it becomes ever more important to safeguard the in- tegrity of information people consume. News arti- cles about important events in the world can affect political choices. It is therefore crucial to pinpoint what information people know to be trustworthy, factual and unbiased. One way to do this is by using a computational system that detects an au- thor’s or publisher’s bias in a news article. In Pot- thast et al. (2018) we have seen that it is possible to build such a system by relying on the writing characteristics of a text. To shed more light on potential linguistic com- putational methods for hyperpartisan news detec- tion, we present our participation in the SemEval 2019 shared task on hyperpartisan news detection, of which the purpose is to identify whether a news article contains hyperpartisan (Kiesel et al., 2019) content. For our contribution, we set out to ex- periment with various types and levels of features, such as a) sentiment that could indicate an au- thor’s bias (Recasens et al., 2013), b) bias-laden words such as assertives, factives and hedges, and c) part-of-speech (from now on POS) filtered n– grams. In the end, we decided to submit a model that only uses the negative sentiment of an article as a feature. We obtained an accuracy of 0.621 and a f1 score of 0.694 on the by-article test set, which resulted to the 30th place in the competition. On the by-publisher test set, the systems accuracy was 0.589 and its f1 score 0.623 (20th place). 2 Related Work One of the first studies on detecting linguistic bias in online texts were mainly focused on detecting biased language in Wikipedia articles. Despite the domain difference, this task is related to ours be- cause Wikipedia is also a source of information which should contain unbiased language. Systems that were employed for this task used a combina- tion of linguistic features, such as POS n-grams and binary features representing the usage …

Content truncated to 500 words