UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3449 - 3456 July 10-15, 2022 ©2022 Association for Computational Linguistics UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis Fatemehsadat Mireshghallah1∗, Vaishnavi Shrivastava2, Milad Shokouhi2, Taylor Berg-Kirkpatrick1, Robert Sim3, Dimitrios Dimitriadis3 1 University of California San Diego, 2 Microsoft Corporation, 3 Microsoft Research [fatemeh, tberg]@ucsd.edu, [vashri,milads,rsim,didimit]@microsoft.com Abstract Global models are typically trained to be as gen- eralizable as possible. Invariance to the specific user is considered desirable since models are shared across multitudes of users. However, these models are often unable to produce personalized responses for individual users, based on their data. Contrary to widely-used personalization techniques based on few-shot and meta-learning, we propose UserIdentifier, a novel scheme for training a single shared model for all users. Our approach produces personalized responses by prepending a fixed, user-specific non-trainable string (called “user identifier”) to each user’s input text. Unlike prior work, this method doesn’t need any additional model parameters, any extra rounds of personal few-shot learning, or any change made to the vocabulary. We empirically study different types of user identifiers (numeric, alphanumeric, and also randomly generated) and demonstrate that, surprisingly, randomly generated user identifiers outperform the prefix- tuning based state-of-the-art approach by up to 13%, on a suite of sentiment analysis datasets. 1 Introduction Personalization arises in applications where dif- ferent clients need models specifically customized to their environment and user profiles (Yang and Eisenstein, 2017; Mazaré et al., 2018; Flek, 2020). This need for customization stems from the inherent heterogeneity existing in the data and the labels, especially when the task is classification (Kulkarni et al., 2020; Wang et al., 2018). Fig. 1 shows an example of the sentence “That is just great!”. This sentence could carry a positive sentiment, a neutral apathetic sentiment, or even a completely negative sentiment. A non-personalized model cannot correctly predict the label for different users. Most techniques for personalization generally involve two phases: first, a shared, global model is ∗ Work done as part of an MSR internship. Corresponding author email: fatemeh@ucsd.edu Activation Text 𝑝! "#$ ℎ! ℎ% ℎ& ℎ' ℎ( Activation 𝑝! )** Text That is just great ! ℎ! ℎ% ℎ& ℎ' ℎ( That is just great ! Activation Text Activation Text anka Sau That is just great! ℎ! ℎ% ℎ& ℎ' ℎ( ℎ+ ℎ, Beh Key That is just great! ℎ! ℎ% ℎ& ℎ' ℎ( ℎ+ ℎ, Prefix Tuning Ours (UserIdentifier) User (𝑢) Input (𝑥) Random user identifier Random user identifier Figure 1: An overview of the proposed method, UserIdentifier, compared to its prefix-tuning counterpart. pkat 1 , pbee 1 denote the trainable prefix vector for users kat and bee, in the prefix tuning method (Zhong et al., 2021). UserIdentifier, on the other hand, does not have trainable user-specific parameters and uses random per-user (UID) strings (“anka Sau” and “Beh KY”), to condition a shared model, for each user. built between all users, and then, it is …

Content truncated to 500 words