A Computational Framework for Slang Generation

A Computational Framework for Slang Generation Zhewei Sun1, Richard Zemel1,2, Yang Xu1,2 1Department of Computer Science, University of Toronto, Toronto, Canada 2Vector Institute for Artificial Intelligence, Toronto, Canada {zheweisun, zemel, yangxu}@cs.toronto.edu Abstract Slang is a common type of informal lan- guage, but its flexible nature and paucity of data resources present challenges for existing natural language systems. We take an ini- tial step toward machine generation of slang by developing a framework that models the speaker’s word choice in slang context. Our framework encodes novel slang meaning by relating the conventional and slang senses of a word while incorporating syntactic and contextual knowledge in slang usage. We con- struct the framework using a combination of probabilistic inference and neural contrastive learning. We perform rigorous evaluations on three slang dictionaries and show that our approach not only outperforms state-of-the-art language models, but also better predicts the historical emergence of slang word usages from 1960s to 2000s. We interpret the pro- posed models and find that the contrastively learned semantic space is sensitive to the simi- larities between slang and conventional senses of words. Our work creates opportunities for the automated generation and interpretation of informal language. 1 Introduction Slang is a common type of informal language that appears frequently in daily conversations, social media, and mobile platforms. The flexible and ephemeral nature of slang (Eble, 1989; Landau, 1984) poses a fundamental challenge for computa- tional representation of slang in natural language systems. As of today, slang constitutes only a small portion of text corpora used in the natural language processing (NLP) community, and it is severely under-represented in standard lexical resources (Michel et al., 2011). Here we propose a novel framework for automated generation of slang with a focus on generative modeling of slang word meaning and choice. Existing language models trained on large-scale text corpora have shown success in a variety of NLP tasks. However, they are typically biased toward formal language and under-represent slang. Consider the sentence ‘‘I have a feeling he’s gonna himself someday’’. Directly apply- ing a state-of-the-art GPT-2 (Radford et al., 2019) based language infilling model (e.g., Donahue et al., 2020) would result in the retrieval of kill as the most probable word choice (probability = 7.7%). However, such a language model is lim- ited and near-insensitive to slang usage, for exam- ple, ice—a common slang alternative for kill— received virtually 0 probability, suggesting that existing models of distributional semantics, even the transformer-type models, do not capture slang effectively, if at all. Our goal is to extend the capacity of NLP systems toward slang in a principled framework. As an initial step, we focus on modeling the gen- erative process of slang, specifically the problem of slang word choice that we illustrate in Figure 1. Given an intended slang sense such as ‘‘to kill’’, we ask how we can emulate the speaker’s choice of slang word(s) in informal context.1 We are particularly interested in how the speaker chooses existing words from the lexicon and makes inno- …

Content truncated to 500 words