Ideology Prediction from Scarce and Biased Supervision: Learn to Disregard the “What” and Focus on the “How”!

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 9529–9549 July 9-14, 2023 ©2023 Association for Computational Linguistics Ideology Prediction from Scarce and Biased Supervision: Learn to Disregard the “What” and Focus on the “How”! Chen Chen School of Management and Economics CUHK Shenzhen, China chenchen2020@cuhk.edu.cn Dylan Walker Argyros School of Business and Economics Chapman University Dylan@chapman.edu Venkatesh Saligrama College of Engineering Boston University srv@bu.edu Abstract We propose a novel supervised learning ap- proach for political ideology prediction (PIP) that is capable of predicting out-of-distribution inputs. This problem is motivated by the fact that manual data-labeling is expensive, while self-reported labels are often scarce and exhibit significant selection bias. We propose a novel statistical model that decomposes the document embeddings into a linear superposition of two vectors; a latent neutral context vector indepen- dent of ideology, and a latent position vector aligned with ideology. We train an end-to-end model that has intermediate contextual and po- sition vectors as outputs. At deployment time, our model predicts labels for input documents by exclusively leveraging the predicted posi- tion vectors. On two benchmark datasets we show that our model is capable of outputting predictions even when trained with as little as 5% biased data, and is significantly more accu- rate than the state-of-the-art. Through crowd- sourcing we validate the neutrality of contex- tual vectors, and show that context filtering results in ideological concentration, allowing for prediction on out-of-distribution examples. 1 Introduction Political Ideology Prediction (PIP) is motivated by a number of applications such as the need to under- stand government’s policy making (Potrafke, 2018; Hunt, 2009), a legislator’s partisan/non-partisan actions (Zingher, 2014; Nice, 1985; Berry et al., 2007), the general public’s sentiment support for legislation (Berry et al., 2007; Budge et al., 1987; Rudolph, 2009) etc. Our overall goal is to infer ideology of legisla- tors or the general public from written texts, media posts, and speeches. We can train a PIP model with ground truth labels (e.g., liberal vs conservative) with a standard supervised learning approach. In some cases, such as for legislators, ground-truth labels can be inferred through party affiliation in- formation, and these have served as good proxies for ideology (Poole and Rosenthal, 2001). Selection Bias. In other situations, such as, for the domain of social media posts, ground truth is diffi- cult to obtain, and is based on self-reported affilia- tion of the users (Bakshy et al., 2015). Such self- reported affiliations are generally sparse, and when reported tend to be of extreme polarity (Conover et al., 2011). While manual labeling by annota- tors (e.g., AMTs) (Preo¸tiuc-Pietro et al., 2017) can be leveraged, selection bias is still an issue due to oversampling of posts from extreme vocal posters (Cohen and Ruths, 2013; Mustafaraj et al., 2011; Kohut and Bowman, 2018; Gao et al., 2015). Justification. A black-box model trained exclu- sively on scarce and polarized group is likely to perform poorly on the under-observed moderates who are the majority and are …

Content truncated to 500 words