DAPPER: Learning Domain-Adapted Persona Representation Using Pretrained BERT and External Memory

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 643–652 December 4 - 7, 2020. c⃝2020 Association for Computational Linguistics 643 DAPPER: Learning Domain-Adapted Persona Representation Using Pretrained BERT and External Memory Prashanth Vijayaraghavan MIT Media lab pralav@mit.edu Eric Chu MIT Media lab echu@mit.edu Deb Roy MIT Media lab dkroy@media.mit.edu Abstract Research in building intelligent agents have emphasized the need for understanding char- acteristic behavior of people. In order to re- flect human-like behavior, agents require the capability to comprehend the context, infer in- dividualized persona patterns and incremen- tally learn from experience. In this paper, we present a model called DAPPER that can learn to embed persona from natural language and alleviate task or domain-specific data sparsity issues related to personas. To this end, we implement a text encoding strategy that lever- ages a pretrained language model and an ex- ternal memory to produce domain-adapted per- sona representations. Further, we evaluate the transferability of these embeddings by simu- lating low-resource scenarios. Our compara- tive study demonstrates the capability of our method over other approaches towards learn- ing rich transferable persona embeddings. Em- pirical evidence suggests that the learnt per- sona embeddings can be effective in down- stream tasks like hate speech detection. 1 Introduction With increasing human-machine hybrid technolo- gies, the real-world interactions with AI systems are often stilted. This shortcoming can be attributed to the lack of shared common knowledge about how people will act, communicate and react un- der different circumstances. Several studies in the field of psychology (Goldberg, 1990; Barrick and Mount, 1993, 1991) have established the role of personas in governing how people process informa- tion, attend to and interpret life-experiences, and respond to social situations. Specifically, the rela- tionship between personality and natural language have been widely studied (Digman and Takemoto- Chock, 1981; Pennebaker et al., 2003). For ex- ample, a narcissistic person might make frequent use of first-person expressions (I, me, myself, for me, etc.). Therefore, endowing machines with the persona information can lead to the develop- ment of psychologically plausible intelligent sys- tems. Though computational models of personality have generally followed prior psychological mod- els or theories (Hjelle and Ziegler, 1992; Costa and PAUL, 1996), multiple definitions of personas have been in use depending on the nature of the do- main or task at hand. There has been considerable amount of interest in the past that used NLP tools to conduct persona analysis of fictional charac- ters in literary texts (Flekova and Gurevych, 2015; Mairesse et al., 2007). Motivated by such works, we focus on deriving persona representations that explain human social behavior categorized accord- ing to their influences on language, conversations and actions in different social contexts. In this work, we define persona as the sum total of mental, emotional, and social characteristics of an individual (Soloff, 1985). This broad definition, while basing on theoretical foundations, allows us to learn persona embeddings from annotated …

Content truncated to 500 words