Tintin at SemEval-2019 Task 4: Detecting Hyperpartisan News Article with only Simple Tokens

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1062–1066 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1062 Tintin at SemEval-2019 Task 4: Detecting Hyperpartisan News Article with only Simple Tokens Yves Bestgen Centre for English Corpus Linguistics Universit´e catholique de Louvain Place Cardinal Mercier, 10 1348 Louvain-la-Neuve, Belgium yves.bestgen@uclouvain.be Abstract Tintin, the system proposed by the CECL for the Hyperpartisan News Detection task of Se- mEval 2019, is exclusively based on the tokens that make up the documents and a standard su- pervised learning procedure. It obtained very contrasting results: poor on the main task, but much more effective at distinguishing docu- ments published by hyperpartisan media out- lets from unbiased ones, as it ranked first. An analysis of the most important features high- lighted the positive aspects, but also some po- tential limitations of the approach. 1 Introduction This report presents the participation of Tintin (Centre for English Corpus Linguistics) in Task 4 of SemEval 2019 entitled Hyperpartisan News Detection. This task is defined as follows by the organizers1: ”Given a news article text, decide whether it follows a hyperpartisan argumentation, i.e., whether it exhibits blind, prejudiced, or un- reasoning allegiance to one party, faction, cause, or person.” This question is related to the detection of fake news, a hot topic in our internet and social me- dia world (P´erez-Rosas et al., 2017). There are, however, essential differences between these two tasks. An article can be hyperpartisan without mentioning any fake content. Another difference is that it is a news article (or even a claim) that is fake whereas a news article but also a media outlet (or publisher) can be considered as hyperpartisan. The challenge organizers took these two possibili- ties (i.e. an article or a publisher can be hyperpar- tisan) into account by offering two test sets. The main test set, the labels-by-article one, contained documents that had been assessed as hyperparti- san or not by human judges, while the documents 1https://pan.webis.de/semeval19/semeval19-web in the secondary test set, the labels-by-publisher one, had been categorized according to whether their publishers were considered to be hyperpar- tisan or not by organizations that disseminate this type of evaluation. In both these test sets, partici- pants had to decide whether a document expresses a hyperpartisan point-of-view or not. If the main task is particularly interesting, the secondary task is also relevant because it is about achieving through an automatic procedure what a series of organizations manually perform in a way that is sometimes called into question as to its im- partiality and quality (Wilner, 2018). However, in this context, the task would preferably be evalu- ated, not at the document level, but at the publisher level by providing several documents from a pub- lisher and asking whether the publisher is biased or not. Nevertheless, it can be assumed that many systems developed for categorizing publishers will start by evaluating each document separately and thus getting good performance in the current sec- ondary task is at …

Content truncated to 500 words