TWEETSUM: Event oriented Social Summarization Dataset

Proceedings of the 28th International Conference on Computational Linguistics, pages 5731–5736 Barcelona, Spain (Online), December 8-13, 2020 5731 TWEETSUM: Event-oriented Social Summarization Dataset Ruifang He Liangliang Zhao Huanyu Liu School of Computer Science and Technology Tianjin University Tianjin, China {rfhe, liangliangzhao, huanyuliu}@tju.edu.cn Abstract With social media becoming popular, a vast of short and noisy messages are produced by mil- lions of users when a hot event happens. Developing social summarization systems becomes more and more critical for people to quickly grasp core and essential information. However, the publicly available and high-quality large scale dataset under social media situation is rare. Constructing such corpus is not easy and very expensive since short texts have very complex social characteristics. Though there exist some datasets, they only consider the text on social media and ignore the potential user relations relevant signals on social network. In this paper, we construct TWEETSUM, a new event-oriented dataset for social summarization. The original data is collected from twitter and contains 12 real world hot events with a total of 44,034 tweets and 11,240 users. We create expert summaries for each event, and we also have the annotation quality evaluation. In addition, we collect additional social signals (i.e. user relations, hashtags and user profiles) and further establish user relation network for each event. To our knowledge, it is the first event-oriented social summarization dataset that contains social relationships. Besides the detailed dataset description, we show the performance of several typical extractive summa- rization methods on TWEETSUM to establish baselines. For further researches, we will release this dataset to the public. 1 Introduction Social media has become an important real-time information source, especially during emergencies, natural disasters and other hot events. According to a new Pew Research Center survey, social media has surpassed traditional news platforms (such as TV and radio) as a news source for Americans: about two- thirds of American adults (68%) get news via social media. Among all major social media sites, Twitter is still the site Americans most commonly use for news, with 71% of Twitter’s users get their news from Twitter. However, it can often be daunting to catch up with the most recent contents due to high volume and velocity of tweets. Hence, social summarization aiming to acquire the most representative and concise information from massive tweets when a hot event happens is particularly urgent. In recent years, many large-scale summarization datasets have been proposed such as New York Times (Sandhaus, 2008), Gigaword (Napoles et al., 2012), NEWSROOM (Grusky et al., 2018) and CNN/DAILYMAIL (Nallapati et al., 2016). However, most of these datasets focus on formal document summarization. Actually, social media text has many different characteristics from formal documents: 1) Short. The length of a tweet is limited to 140 characters, which is much shorter than formal document. 2) Informal. Tweets usually contains informal expressions such as abbreviations, typos, special symbols and so on, which make tweets more difficult to deal with. 3) Social signal. There are different kinds of social signals on …

Content truncated to 500 words