The Sally Smedley Hyperpartisan News Detector at SemEval-2019 Task 4

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1057–1061 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1057 The Sally Smedley Hyperpartisan News Detector at SemEval-2019 Task 4: Learning Classifiers with Feature Combinations and Ensembling Kazuaki Hanawa*1,2, Shota Sasaki*1,2, Hiroki Ouchi1,2, Jun Suzuki2,1, Kentaro Inui2,1 (* equal contribution) 1RIKEN AIP, 2Tohoku University {hanawa, sasaki.shota, hiroki.ouchi, jun.suzuki, inui} @ecei.tohoku.ac.jp Abstract This paper describes our system submitted to the formal run of SemEval-2019 Task 4: Hy- perpartisan news detection. Our system is based on a linear classifier using several fea- tures, i.e., 1) embedding features based on the pre-trained BERT embeddings, 2) arti- cle length features, and 3) embedding fea- tures of informative phrases extracted from the by-publisher dataset. Our system achieved 80.9% accuracy on the test set for the formal run and got the 3rd place out of 42 teams. 1 Introduction Hyperpartisan news detection (Kiesel et al., 2019; Potthast et al., 2018) is a binary classification task, in which given a news article text, systems have to decide whether or not it follows a hyperparti- san argumentation, i.e., “whether it exhibits blind, prejudiced, or unreasoning allegiance to one party, faction, cause, or person” (2019). As resources for building such a system, the by-publisher and by-article datasets are provided by the organizer. The by-publisher dataset is a col- lection of news articles labeled with the over- all bias of the publisher as provided by Buz- zFeed journalists or MediaBiasFactCheck.com. The by-article dataset is a collection labeled through crowdsourcing on an article basis. This data contains only the articles whose labels are agreed by all the crowd-workers. The performance measure is accuracy on a balanced set of articles. Our system is based on a linear classifier using several types of features mainly consist- ing of 1) embedding features based on the pre- trained BERT embeddings (Devlin et al., 2018) and 2) article length features and 3) embedding features of informative phrases extracted from by-publisher dataset. Our system achieved 80.9% accuracy on the test set for the formal run and got 3rd place out of 42 teams in the formal run. 2 System Description This section first presents an overview of our sys- tem and then elaborate on the feature set. 2.1 Overview of Our System Our system is based on a linear classifier that mod- els the conditional probability distribution over the two labels (positive or negative) given features. Let f be a feature vector. W denotes a trainable weight matrix, and b is a trainable bias vector, where f ∈ RD, W ∈ RD×2 and b ∈ R2, respec- tively. Then, we compute the conditional proba- bility as follows: y = softmax(W⊤f + b). (1) where, softmax(·) represent the softmax function that receives an N-dimensional vector x and re- turns another N dimensional vector, namely: softmax(x) = �exp(x) i exp(xi), (2) and x = (x1, . . . , xN)⊤. After the softmax com- putation, we obtain the two-dimensional vector y ∈ R2. We assume …

Content truncated to 500 words