UBC-NLP at SemEval-2019 Task 4: Hyperpartisan News Detection With Attention-Based Bi-LSTMs

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1072–1077 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1072 UBC-NLP at SemEval-2019 Task 4: Hyperpartisan News Detection With Attention-Based Bi-LSTMs Chiyu Zhang Arun Rajendran Muhammad Abdul-Mageed Natural Language Processing Lab The University of British Columbia chiyu94@alumni.ubc.ca, arun95@math.ubc.ca, muhammad.mageeed@ubc.ca Abstract We present our deep learning models sub- mitted to the SemEval-2019 Task 4 compe- tition focused at Hyperpartisan News Detec- tion. We acquire best results with a Bi-LSTM network equipped with a self-attention mecha- nism. Among 33 participating teams, our sub- mitted system ranks top 7 (65.3% accuracy) on the labels-by-publisher sub-task and top 24 out of 44 teams (68.3% accuracy) on the labels-by-article sub-task (65.3% accuracy). We also report a model that scores higher than the 8th ranking system (78.5% accuracy) on the labels-by-article sub-task. 1 Introduction Spread of fake news (e.g., Allcott and Gentzkow (2017); Horne and Adali (2017)) (or ‘low-quality’ information (Qiu et al., 2017), among other terms) can have destructive economic impacts (San- doval, 2008), result in dangerous real world con- sequences (Akpan, 2016), or possibly undermine the very democratic bases of modern societies (Qiu et al., 2017; Allcott and Gentzkow, 2017). Several approaches have been employed for de- tecting fake stories online, including detecting the sources that are highly polarized (or hyperparti- san)] (Potthast et al., 2017). Detecting whether a source is extremely biased for or against a given party can be an effective step toward identifying fake news. Most research on news orientation prediction employed machine learning methods based on fea- ture engineering. For example, Pla and Hurtado (2014) use features such as text n-grams, part-of- speech tags, hashtags, etc. with an SVM classifier to tackle political tendency identification in twit- ter. Potthast et al. (2017) investigate the writing style of hyperpartisan and mainstream news using a random forest classifier (Koppel et al., 2007). Further, Preot¸iuc-Pietro et al. (2017) use a linear regression algorithm to categorize Twitter users into a fine-grained political group. The authors were able to show a relationship between language use and political orientation. Nevertheless, previous works have not consid- ered the utility of deep learning methods for hy- perpartisanship detection. Our goal is to bridge this gap by investigating the extent to which deep learning can fare on the task. More pre- cisely, we employ several neural network architec- tures for hyperpartisans news detection, including long short-term memory networks (LSTM), con- volutional neural networks (CNN), bi-directional long short term memory networks (Bi-LSTM), convolutional LSTM (CLSTM), recurrent convo- lutional neural network (RCNN), and attention- based LSTMs and Bi-LSTMs. We make the following contributions: (1) we in- vestigate the utility of several deep learning mod- els for classifying hyperpartisan news, (2) we test model performance under a range of training set conditions to identify the impact of training data size on the task, and (3) we probe our models with an attention mechanism coupled with a simple vi- sualization method to discover meaningful contri- butions of various lexical …

Content truncated to 500 words