A French Corpus for Event Detection on Twitter

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6220–6227 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6220 A French Corpus for Event Detection on Twitter B´eatrice Mazoyer1,3 , Julia Cag´e2 , Nicolas Herv´e3 , C´eline Hudelot1 1CentraleSup´elec (Paris-Saclay University), MICS, Gif-sur-Yvette, France 2SciencesPo Paris, Department of Economics, Paris, France 3Institut National de l’Audiovisuel, Research Department, Bry-sur-Marne, France {beatrice.mazoyer, celine.hudelot}@centralesupelec.fr, julia.cage@sciencespo.fr, nherve@ina.fr Abstract We present Event2018, a corpus annotated for event detection tasks, consisting of 38 million tweets in French (retweets excluded) including more than 130,000 tweets manually annotated by three annotators as related or unrelated to a given event. The 257 events were selected both from press articles and from subjects trending on Twitter during the annotation period (July to August 2018). In total, more than 95,000 tweets were annotated as related to one of the selected events. We also provide the titles and URLs of 15,500 news articles automatically detected as related to these events. In addition to this corpus, we detail the results of our event detection experiments on both this dataset and another publicly available dataset of tweets in English. We ran extensive tests with different types of text embeddings and a standard Topic Detection and Tracking algorithm, and detail our evaluation method. We show that tf-idf vectors allow the best performance for this task on both corpora. These results are intended to serve as a baseline for researchers wishing to test their own event detection systems on our corpus. Keywords: Natural Language Processing, Twitter, Topic Detection and Tracking, Event detection, Dataset, French 1. Introduction Twitter is known as a major source of information about public events: a recent study by McGregor and Molyneux (2018) shows that journalists using Twitter as part of their daily work consider tweets as newsworthy as headlines from the Associated Press. Twitter can thus help re- searchers understand public opinion, measure the interest raised by certain types of topics, or detect some unpre- dictable events in real time. Indeed, the social network has been used for flood prevention (de Bruijn et al., 2017), to detect local events (Wei et al., 2019), or to predict stock market movements (Pagolu et al., 2016), among many other applications. However, the specificities of Twitter (short texts, use of slang, abbreviations, hashtags, images and videos, very high volume of data) make all automatic detection tasks very difficult on tweet datasets. Besides, research on general topic detection and tracking (without specification of the type of topic) lacks some publicly available tweet datasets to produce reproducible results. This is all the more true for non-English languages. Existing datasets may also have different definitions of event or topic, which leads to issues when comparing event-detection systems. Many works on event detection are actually focused on burst de- tection (detecting topics such as natural disasters, attacks, etc., that cause an unusual volume of tweets), and do not at- tempt to assess the relative size of events. We seek to detect …

Content truncated to 500 words