The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 175–185, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics The effect of wording on message propagation: Topic- and author-controlled natural experiments on Twitter Chenhao Tan Dept. of Computer Science Cornell University chenhao@cs.cornell.edu Lillian Lee Dept. of Computer Science Cornell University llee@cs.cornell.edu Bo Pang Google Inc. bopang42@gmail.com Abstract Consider a person trying to spread an important message on a social network. He/she can spend hours trying to craft the message. Does it actually matter? While there has been extensive prior work look- ing into predicting popularity of social- media content, the effect of wording per se has rarely been studied since it is of- ten confounded with the popularity of the author and the topic. To control for these confounding factors, we take advantage of the surprising fact that there are many pairs of tweets containing the same url and written by the same user but employing different wording. Given such pairs, we ask: which version attracts more retweets? This turns out to be a more difficult task than predicting popular topics. Still, hu- mans can answer this question better than chance (but far from perfectly), and the computational methods we develop can do better than both an average human and a strong competing method trained on non- controlled data. 1 Introduction How does one make a message “successful”? This question is of interest to many entities, including political parties trying to frame an issue (Chong and Druckman, 2007), and individuals attempting to make a point in a group meeting. In the first case, an important type of success is achieved if the national conversation adopts the rhetoric of the party; in the latter case, if other group members repeat the originating individual’s point. The massive availability of online messages, such as posts to social media, now affords re- searchers new means to investigate at a very large scale the factors affecting message propagation, also known as adoption, sharing, spread, or vi- rality. According to prior research, important fea- tures include characteristics of the originating au- thor (e.g., verified Twitter user or not, author’s messages’ past success rate), the author’s social network (e.g., number of followers), message tim- ing, and message content or topic (Artzi et al., 2012; Bakshy et al., 2011; Borghol et al., 2012; Guerini et al., 2011; Guerini et al., 2012; Hansen et al., 2011; Hong et al., 2011; Lakkaraju et al., 2013; Milkman and Berger, 2012; Ma et al., 2012; Petrovi´c et al., 2011; Romero et al., 2013; Suh et al., 2010; Sun et al., 2013; Tsur and Rappoport, 2012). Indeed, it’s not surprising that one of the most retweeted tweets of all time was from user BarackObama, with 40M followers, on November 6, 2012: “Four more years. [link to photo]”. Our interest in this paper is the effect of alterna- tive message wording, meaning how the message is said, rather than what the message is about. In contrast to the …

Content truncated to 500 words