Controlling Human Perception of Basic User Traits

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2335–2341 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Controlling Human Perception of Basic User Traits Daniel Preot¸iuc-Pietro and Sharath Chandra Guntuku and Lyle Ungar Positive Psychology Center University of Pennsylvania {danielpr@sas,sharathg@sas,ungar@cis}.upenn.edu Abstract Much of our online communication is text- mediated and, lately, more common with automated agents. Unlike interacting with humans, these agents currently do not tai- lor their language to the type of person they are communicating to. In this pilot study, we measure the extent to which hu- man perception of basic user trait infor- mation – gender and age – is controllable through text. Using automatic models of gender and age prediction, we estimate which tweets posted by a user are more likely to mis-characterize his traits. We perform multiple controlled crowdsourc- ing experiments in which we show that we can reduce the human prediction accuracy of gender to almost random – an over 20% drop in accuracy. Our experiments show that it is practically feasible for multiple applications such as text generation, text summarization or machine translation to be tailored to specific traits and perceived as such. 1 Introduction Advances in Natural Language Processing are leading to a point when text generation methods are deployed at scale. However, in the quest to make these applications more likable, effective and hence more usable, these methods should con- sider a way to adapt themselves to the person or type of persons they are interacting with (Bates, 1994; Loyall and Bates, 1997) e.g., a student may learn better from a tutoring agent that expresses similar traits to himself (Baylor and Kim, 2004). In this study, we explore the feasibility of con- trolling human perception of traits using auto- mated methods. Flekova et al. (2016); Carpenter et al. (2016) are the first to study the difference be- tween user traits and their perception by external raters using tweets from social media. Their focus was on quantifying differences between percep- tion and reality and analyzing text features which lead to mis-perception. This study goes a step fur- ther, and using the same experimental design and crowdsourcing, aims to use automatic methods to control human perception of basic user traits – here age and gender – through tweets. To this end, we use gender and age prediction algorithms to se- lect tweets posted by users with a known trait with the goal of increasing or decreasing human rater accuracy in guessing their traits. Obfuscating gender as identified by an automatic classifier was attempted in (Reddy and Knight, 2016). This problem is related, but very differ- ent to ours as we study human perception which is both different (Flekova et al., 2016) and more complex. Reddy and Knight (2016) study a range of lexical substitutions that can be performed in or- der to decrease the prediction accuracy of a classi- fier, although acknowledging that these may affect lexical coherence. In this pilot study, we circum- vent this …

Content truncated to 500 words