Identifying Dogmatism in Social Media: Signals and Models

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 690–699, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Identifying Dogmatism in Social Media: Signals and Models Ethan Fast and Eric Horvitz ethaen@stanford.edu, horvitz@microsoft.com Abstract We explore linguistic and behavioral features of dogmatism in social media and construct statistical models that can identify dogmatic comments. Our model is based on a corpus of Reddit posts, collected across a diverse set of conversational topics and annotated via paid crowdsourcing. We operationalize key aspects of dogmatism described by existing psychol- ogy theories (such as over-confidence), find- ing they have predictive power. We also find evidence for new signals of dogmatism, such as the tendency of dogmatic posts to refrain from signaling cognitive processes. When we use our predictive model to analyze millions of other Reddit posts, we find evidence that suggests dogmatism is a deeper personality trait, present for dogmatic users across many different domains, and that users who engage on dogmatic comments tend to show increases in dogmatic posts themselves. 1 Introduction “I’m supposed to trust the opinion of a MS min- ion? The people that produced Windows ME, Vista and 8? They don’t even understand people, yet they think they can predict the behavior of new, self- guiding AI?” –anonymous “I think an AI would make it easier for Patients to confide their information because by nature, a robot cannot judge them. Win-win? :D”’ –anonymous Dogmatism describes the tendency to lay down opinions as incontrovertibly true, without respect for conflicting evidence or the opinions of others (Ox- ford Dictionary, 2016). Which user is more dog- matic in the examples above? This question is sim- ple for humans. Phrases like “they think” and “they don’t even understand,” suggest an intractability of opinion, while “I think” and “win-win?” suggest the opposite. Can we train computers to draw sim- ilar distinctions? Work in psychology has called out many aspects of dogmatism that can be modeled computationally via natural language, such as over- confidence and strong emotions (Rokeach, 1954). We present a statistical model of dogmatism that addresses two complementary goals. First, we val- idate psychological theories by examining the pre- dictive power of feature sets that guide the model’s predictions. For example, do linguistic signals of certainty help to predict a post is dogmatic, as the- ory would suggest? Second, we apply our model to answer four questions: R1: What kinds of topics (e.g., guns, LGBT) at- tract the highest levels of dogmatism? R2: How do dogmatic beliefs cluster? R3: How does dogmatism influence a conversa- tion on social media? R4: How do other user behaviors (e.g., frequency and breadth of posts) relate to dogmatism? We train a predictive model to classify dogmatic posts from Reddit, one of the most popular discus- sion communities on the web.1 Posts on Reddit cap- ture discussion and debate across a diverse set of do- mains and topics – users talk about everything from climate change and abortion, to world news and …

Content truncated to 500 words