Automatic Classification of Tweets for Analyzing Communication Behavior of Museums

Automatic Classification of Tweets for Analyzing Communication Behavior of Museums Nicolas Foucault 1, 2, Antoine Courtin 3, 4 1 UMR 7114 – Modèles, Dynamiques, Corpus, Nanterre, France 2 Julie Desk, Paris, France 3 Labex "Les Passés dans le Présent", Nanterre, France 4 Institut National d’Histoire de l’Art, Paris, France nicolas.foucault@juliedesk.com, antoine.courtin@mac.com Abstract In this paper, we present a study on tweet classification which aims to define the communication behavior of the 103 French museums that participated in 2014 in the Twitter operation: MuseumWeek. The tweets were automatically classified in four communication cate- gories: sharing experience, promoting participation, interacting with the community, and promoting-informing about the institution. Our classification is multi-class. It combines Support Vector Machines and Naive Bayes methods and is supported by a selection of eighteen subtypes of features of four different kinds: metadata information, punctuation marks, tweet-specific and lexical features. It was tested against a corpus of 1,095 tweets manually annotated by two experts in Natural Language Processing and Information Communication and twelve Community Managers of French museums. We obtained an state-of-the-art result of F1-score of 72% by 10-fold cross- validation. This result is very encouraging since is even better than some state-of-the-art results found in the tweet classification literature. Keywords: NLP, classification, tweets, communication, behavior, museums, annotation, community managers, corpus, Twitter, MuseumWeek. 1. Introduction In the last decade, the two American companies, Facebook and Twitter, have captured the lion’s share of the competi- tive international market built around the creative and par- ticipatory practices of social media. Their business model is based on the expectations and participation of audiences. Beyond studying the behavior of audiences, social media call for a deeper analysis of the actions of profession- als, such as Community Managers (CMs) of museums, who use social networks for promotional/institutional pur- poses, in order to better understand their practices and in- tentions Johnson et al. (2012). To do so, it is essential to achieve a fine-grained classification of the communica- tion content of their messages in micro-blogs such as those found on Twitter, on a large-scale. The work presented in this paper was conducted with this aim in view and is part of the NOS project1. NOS aims at providing CMs of French cultural institutions with a tool dedicated to help them in the analysis of social-media mes- sages sent by other institutions or users. 2. Literature Overview Tweet analysis has led to a large number of studies in many domains such as ideology prediction in Information Sci- ences (Djemili et al., 2014), spam detection in Security (Ya- masaki, 2011), dialog analysis in Linguistics (Boyd et al., 2010), and natural disaster anticipation in Emergency (Gel- ernter and Mushegian, 2011; Sakaki et al., 2013). Complementary efforts have been made in Social Sci- ences and Digital Humanities to develop tweet classifica- tions (Dann, 2010; Riemer and Richter, 2010; Shiri and Rathi, 2013; Stvilia and Gibradze, 2014). 1 bit.ly/1Z5VI7x However, few studies aim at classifying tweets according to communication classes. They mostly rely on small ref- erence sets analyzed …

Content truncated to 500 words