Characterizing Social Spambots by their Human Traits

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 5148–5158 August 1–6, 2021. ©2021 Association for Computational Linguistics 5148 Characterizing Social Spambots by their Human Traits Salvatore Giorgi University of Pennsylvania sgiorgi@sas.upenn.edu Lyle Ungar University of Pennsylvania ungar@cis.upenn.edu H. Andrew Schwartz Stony Brook University has@cs.stonybrook.edu Abstract Social spambots, an emerging class of spam- mers attempting to emulate people, are dif- ficult for both human annotators and classic bot detection techniques to reliably distinguish from genuine accounts. We examine this hu- man emulation through studying the human characteristics (personality, gender, age, emo- tions) exhibited by social spambots’ language, hypothesizing the values for these attributes will be unhuman-like (e.g. unusually high or low). We found our hypothesis mostly discon- firmed — individually, social bots exhibit very human-like attributes. However, a striking pat- tern emerged when consider the full distribu- tions of these estimated human attributes: so- cial bots were extremely similar and average in their expressed personality, demographics, and emotion (in contrast with traditional bots which we found to exhibit more variance and extreme values than genuine accounts). We thus consider how well social bots can be iden- tified only using the 17 variables of these hu- man attributes and ended up with a new state of the art in social spambot detection (e.g. F1 = .946). Further, simulating the situa- tion of not knowing the bots a priori, we found that even an unsupervised clustering using the same 17 attributes could yield nearly as accu- rate of social bot identification (F1 = 0.925). 1 Introduction A social spambot is “a computer algorithm that automatically produces content and interacts with humans on social media, trying to emulate and pos- sibly alter their behavior” (Ferrara et al., 2016). Previous studies have shown that standard spambot detection algorithms as well as human annotators, while quite effective at detecting standard spam (e.g. sales advertisements), fail to accurately distin- guish these human emulating social spambots from genuine Twitter accounts (Cresci et al., 2017). In light of the goal of social spambots to emulate human behavior, we test the assumption that social spambots should not behave differently than human accounts and that a shared set of traits should be common among both groups. Thus, in this study we attempt to characterize and classify social spam- bots, in comparison to genuine Twitter accounts and traditional spambot accounts, by real human traits and states. We estimate demographics (age and gender), personality traits (openness to expe- rience, conscientiousness, extraversion, agreeable- ness, and neuroticism), sentiment (positive and neg- ative), and emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) using a va- riety of pre-trained models. First and foremost, we describe the distributions of these 17 human attributes, and then we evaluate them as features in relatively simple spambot detection classifiers (random forest) and compare to more sophisticated state of the art detection methods. Contributions Our contributions: (1) We charac- terize social spambots in terms of estimated human traits: demographics, personality, sentiment, and emotion. We show that social spambots …

Content truncated to 500 words