Crowdsourcing a Large Corpus of Clickbait on Twitter

Proceedings of the 27th International Conference on Computational Linguistics, pages 1498–1507 Santa Fe, New Mexico, USA, August 20-26, 2018. 1498 Crowdsourcing a Large Corpus of Clickbait on Twitter Martin Potthast1 Tim Gollub2 Kristof Komlossy2 Sebastian Schuster2 Matti Wiegmann2,3 Erika Patricia Garces Fernandez2 Matthias Hagen4 Benno Stein2 1Leipzig University 2Bauhaus-Universität Weimar 3German Aerospace Center (DLR) 4Martin-Luther-Universität Halle-Wittenberg Abstract Clickbait has become a nuisance on social media. To address the urging task of clickbait de- tection, we constructed a new corpus of 38,517 annotated Twitter tweets, the Webis Clickbait Corpus 2017. To avoid biases in terms of publisher and topic, tweets were sampled from the top 27 most retweeted news publishers, covering a period of 150 days. Each tweet has been an- notated on 4-point scale by five annotators recruited at Amazon’s Mechanical Turk. The corpus has been employed to evaluate 12 clickbait detectors submitted to the Clickbait Challenge 2017. Download: https://webis.de/data/webis-clickbait-17.html Challenge: https://clickbait-challenge.org 1 Introduction For publishers of online content, the most popular revenue model is advertising. On freely accessible web pages ads are displayed along the content, and publishers earn fees for each ad seen by the visitors of these pages. Online content is often expected “to pay for itself:” it has to attract enough visitors such that the earned fees exceed the investment made in creating it. Assuming that each piece of content has its target audience, generating profit boils down to letting the respective target audience know where to find it. Ironically, this requires the content itself to be advertised. Notwithstanding other means of advertisement, today, most publishers advertise their content on social media. By spreading “teaser messages” (be it with or without paying promotion fees), publishers engage in viral marketing. Teaser messages consist of two or three parts: (1) a short text, (2) an optional media attachment such as an image or a video, and (3) a link to the publisher’s page where the advertised content is found. From a publisher’s perspective, the ideal teaser message discloses its advertised content to the least possible extent, so that its target audience (better: everyone) is tempted to visit the publisher’s pages. Conversely, from a reader’s perspective, the ideal teaser message is a self-contained summary, so that visiting the publisher’s page is subject only to cases where not everything important could be fitted into the summary, or where the reader wants to learn more about the backgrounds. The two ideals are naturally at odds, and the social network operators may need to reconcile between them. Figure 1 shows examples of teaser messages in the above sense, advertising news content on Twitter. The left-to-right ordering of the messages in the figure is indicative of the degree to which an average reader might feel sufficiently informed without clicking the link (left), less informed but rather curious and urged to click (right), or something in-between (middle). The messages more to the right side of the figure are examples of what is called “clickbait” today. Clickbait is an umbrella term that encompasses all kinds …

Content truncated to 500 words