Team Kit Kittredge at SemEval-2019 Task 4: LSTM Voting System

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1021–1025 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1021 Team Kit Kittredge at SemEval-2019 Task 4: LSTM Voting System Rebekah Cramerus Department of Linguistics University of Potsdam Potsdam, Germany rebekah.cramerus@fulbrightmail.org Tatjana Scheffler Department of Linguistics University of Potsdam Potsdam, Germany tatjana.scheffler@uni-potsdam.de Abstract This paper describes the approach of team Kit Kittredge to SemEval 2019 Task 4: Hyper- partisan News Detection. The goal was bi- nary classification of news articles into the cat- egories of “biased” or “unbiased”. We had two software submissions: one a simple bag-of- words model, and the second an LSTM (Long Short Term Memory) neural network, which was trained on a subset of the original dataset selected by a voting system of other LSTMs. This method did not prove much more suc- cessful than the baseline, however, due to the models’ tendency to learn publisher-specific traits instead of general bias. 1 Introduction With the proliferation of online news agencies af- ter the rise of the Internet, access to information about what is going on in the world has never been more widespread. How that information is presented, however, can have a large influence on what conclusions the reader draws from it. Be- ing able to automatically identify hyperpartisan- ship (bias or adherence to one party or faction over others) in a news article would help individuals in their news consumption and potentially result in a better-informed population. As suggested, the challenge approached in this paper is that of hyperpartisan news detection: a binary classification problem (biased or unbiased) with news articles as data. This task can be consid- ered as related to stance detection and in general, sentiment analysis. The challenge was organized as Task 4 for SemEval 2019 (Potthast et al., 2018) (Kiesel et al., 2019). Final submissions were sub- mitted through TIRA, with the test datasets hidden (Potthast et al., 2019). First in this paper, Section 2 includes an in- troduction of the provided dataset and a descrip- tion of preprocessing techniques used for our ap- proach. Section 3 describes the first submitted software, a bag-of-words model. Section 4 contin- ues with our second approach, an LSTM trained on a subset of the original dataset, and a descrip- tion of how that subset was selected. Section 5 presents our results on the test set and Section 6 delves into analysis, presenting potential reasons why the models did not perform very well. 2 Data During the course of the task participants were granted access to different datasets with which to work. A training dataset of 600,000 articles and a val- idation dataset of 150,000 articles were both re- leased to participants. Both sets contain 50% un- biased and 50% biased articles, and of the latter, half are left-biased and the other half right-biased (in terms of their placement on the political spec- trum). Importantly, these articles were all labeled with the overall bias of their publisher, which was obtained by MediaBiasFactCheck.com …

Content truncated to 500 words