Slang Detection and Identification

Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 881–889 Hong Kong, China, November 3-4, 2019. c⃝2019 Association for Computational Linguistics 881 Slang detection and identification Zhengqi Pei1,2 Zhewei Sun3 Yang Xu3 1Engineering Science Program, University of Toronto 2HAETEK Institute of Machine Intelligence, Shenzhen, China 3Department of Computer Science, University of Toronto zhengqi.pei@mail.utoronto.ca {zheweisun, yangxu}@cs.toronto.edu Abstract The prevalence of informal language such as slang presents challenges for natural language systems, particularly in the automatic discov- ery of flexible word usages. Previous work has explored slang in terms of dictionary con- struction, sentiment analysis, word formation, and interpretation, but scarce research has at- tempted the basic problem of slang detection and identification. We examine the extent to which deep learning methods support au- tomatic detection and identification of slang from natural sentences using a combination of bidirectional recurrent neural networks, con- ditional random field, and multilayer percep- tron. We test these models based on a compre- hensive set of linguistic features in sentence- level detection and token-level identification of slang. We found that a prominent fea- ture of slang is the surprising use of words across syntactic categories or syntactic shift (e.g., verb→noun). Our best models detect the presence of slang at the sentence level with an F1-score of 0.80 and identify its exact position at the token level with an F1-Score of 0.50. 1 Introduction Slang, or ‘the language of streets’ (Green, 2015), is a type of informal language consisting of words and expressions shared within specific groups. A hallmark of slang is its expressivity, instantiated in the flexible use of words. For example, the word sick with the conventional sense of “ill” can also denote a positive slang sense of “awesome”, such as “the band’s album is sick”. The expres- sive nature of slang exemplifies its social function, because it provides an effective way of commu- nicating and knowledge-sharing within groups of distinct social identities, such as in the cases of vulgar tongue (Green, 2015) and online language. On the other hand, the flexible nature of slang use can be intriguing for language users, learners, and natural language systems. Here, we ask whether slang can be automatically detected in natural sen- tences, and what linguistic features might distin- guish slang usage from conventional language use. Our problem statement is simple: Given a natu- ral sentence such as “the band’s album is sick”, can machines learn to 1) detect whether slang usage is present or not (i.e., sentence-level detection), and 2) identify the exact position of the slang term in the sentence (i.e., token-level identification). For each of these tasks, our systems should be able to learn to cope with two main categories of slang usage (Dhuliawala et al., 2016): • Newly extended senses: existing words in the lexicon that develop novel slang senses distinct from their conventional senses, e.g., clutch refers to “an act of grasping” in its con- ventional usage, but is later extended to the slang sense of “tense critical situation”. • Newly created words: …

Content truncated to 500 words