Something’s Brewing! Early Prediction of Controversy-causing Posts from Discussion Features

Proceedings of NAACL-HLT 2019, pages 1648–1659 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1648 Something’s Brewing! Early Prediction of Controversy-causing Posts from Discussion Features Jack Hessel Cornell University jhessel@cs.cornell.edu Lillian Lee Cornell University llee@cs.cornell.edu Abstract Controversial posts are those that split the preferences of a community, receiving both significant positive and significant negative feedback. Our inclusion of the word “com- munity” here is deliberate: what is controver- sial to some audiences may not be so to oth- ers. Using data from several different commu- nities on reddit.com, we predict the ulti- mate controversiality of posts, leveraging fea- tures drawn from both the textual content and the tree structure of the early comments that initiate the discussion. We find that even when only a handful of comments are available, e.g., the first 5 comments made within 15 minutes of the original post, discussion features often add predictive capacity to strong content-and- rate only baselines. Additional experiments on domain transfer suggest that conversation- structure features often generalize to other communities better than conversation-content features do. 1 Introduction Controversial content — that which attracts both positive and negative feedback — is not necessar- ily a bad thing; for instance, bringing up a point that warrants spirited debate can improve com- munity health.1 But regardless of the nature of the controversy, detecting potentially controversial content can be useful for both community mem- bers and community moderators. Ordinary users, and in particular new users, might appreciate being warned that they need to add more nuance or qual- ification to their earlier posts.2 Moderators could be alerted that the discussion ensuing from some 1Coser (1956); Jehn (1995); De Dreu and Weingart (2003) discuss how disagreement interacts with group makeup, group-task type, and outcome. Chen and Berger (2013) demonstrate a non-linear relationship between controversy and amount of subsequent discussion. 2We set aside the issue of trolls whose intent is solely to divide a community. content might need monitoring. Alternately, they could draw community attention to issues possi- bly needing resolution: indeed, some sites already provide explicit sorting by controversy. We consider the controversiality of a piece of content in the context of the community in which it is shared, because what is controversial to some audiences may not be so to others (Chen and Berger, 2013; Jang et al., 2017; Basile et al., 2017). For example, we identify “break up” as a controversial concept in the relationships sub- reddit (a subreddit is a subcommunity hosted on the Reddit discussion site), but the same topic is associated with a lack of controversy in the AskWomen subreddit (where questions are posed for women to answer). Similarly, topics that are controversial in one community may simply not be discussed in another: our analysis identifies “crossfit”, a type of workout, as one of the most controversial concepts in the subreddit Fitness. However, while controversial topics may be community-specific, community moderators still may not be able to determine a priori which posts will attract controversy. Many …

Content truncated to 500 words