Talking to the crowd: What do people react to in online discussions?

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2026–2031, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Talking to the crowd: What do people react to in online discussions? Aaron Jaech, Vicky Zayats, Hao Fang, Mari Ostendorf and Hannaneh Hajishirzi Dept. of Electrical Engineering University of Washington {ajaech,vzayats,hfang,ostendor,hannaneh}@uw.edu Abstract This paper addresses the question of how language use affects community reaction to comments in online discussion forums, and the relative importance of the mes- sage vs. the messenger. A new comment ranking task is proposed based on com- munity annotated karma in Reddit discus- sions, which controls for topic and tim- ing of comments. Experimental work with discussion threads from six subred- dits shows that the importance of different types of language features varies with the community of interest. 1 Introduction Online discussion forums are a popular platform for people to share their views about current events and learn about issues of concern to them. Discus- sion forums tend to specialize on different topics, and people participating in them form communi- ties of interest. The reaction of people within a community to comments posted provides an indi- cation of community endorsement of opinions and value of information. In most discussions, the vast majority of comments spawn little reaction. In this paper, we look at whether (and how) language use affects the reaction, compared to the relative im- portance of the author and timing of the post. Early work on factors that appear to influence crowd-based judgments of comments in the Slash- dot forum (Lampe and Resnick, 2004) indicate that timing, starting score, length of the comment, and poster anonymity/reputation appear to play a role (where anonymity has a negative effect). Judging by differences in popularity of various discussion forums, topic is clearly important. Ev- idence that language use also matters is provided by recent work (Danescu-Niculescu-Mizil et al., 2012; Lakkaraju et al., 2013; Althoff et al., 2014; Tan et al., 2014). Teasing these different factors apart, however, is a challenge. The work presented in this paper provides additional insight into this question by controlling for these factors in a dif- ferent way than previous work and by examining multiple communities of interest. Specifically, us- ing data from Reddit discussion forums, we look at the role of author reputation as measured in terms of a karma k-index, and control for topic and tim- ing by ranking comments in a constrained window within a discussion. The primary contributions of this work include findings about the role of author reputation and variation across communities in terms of aspects of language use that matter, as well as the problem formulation, associated data collection, and devel- opment of a variety of features for characterizing informativeness, community response, relevance and mood. 2 Data Reddit1 is the largest public online discussion fo- rum with a wide variety of subreddits, which makes it a good data source for studying how tex- tual content in a discussion impacts the response of the …

Content truncated to 500 words