What to do about bad language on the internet

Proceedings of NAACL-HLT 2013, pages 359–369, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics What to do about bad language on the internet Jacob Eisenstein jacobe@gatech.edu School of Interactive Computing Georgia Institute of Technology Abstract The rise of social media has brought compu- tational linguistics in ever-closer contact with bad language: text that defies our expecta- tions about vocabulary, spelling, and syntax. This paper surveys the landscape of bad lan- guage, and offers a critical review of the NLP community’s response, which has largely fol- lowed two paths: normalization and domain adaptation. Each approach is evaluated in the context of theoretical and empirical work on computer-mediated communication. In addi- tion, the paper presents a quantitative analy- sis of the lexical diversity of social media text, and its relationship to other corpora. 1 Introduction As social media becomes an increasingly important application domain for natural language processing, we encounter language that is substantially different from many benchmark corpora. The following ex- amples are all from the social media service Twitter: • Work on farm Fri. Burning piles of brush WindyFire got out of control. Thank God for good naber He help get undr control Pants- BurnLegWound. (Senator Charles Grassley) • Boom! Ya ur website suxx bro (Sarah Silverman) • ...dats why pluto is pluto it can neva b a star (Shaquille O’Neil) • michelle obama great. job. and. whit all my. respect she. look. great. congrats. to. her. (Ozzie Guillen) These examples are selected from celebrities (for privacy reasons), but they contain linguistic chal- lenges that are endemic to the medium, including non-standard punctuation, capitalization, spelling, vocabulary, and syntax. The consequences for lan- guage technology are dire: a series of papers has detailed how state-of-the-art natural language pro- cessing (NLP) systems perform significantly worse on social media text. In part-of-speech tagging, the accuracy of the Stanford tagger (Toutanova et al., 2003) falls from 97% on Wall Street Journal text to 85% accuracy on Twitter (Gimpel et al., 2011). In named entity recognition, the CoNLL-trained Stan- ford recognizer achieves 44% F-measure (Ritter et al., 2011), down from 86% on the CoNLL test set (Finkel et al., 2005). In parsing, Foster et al. (2011) report double-digit decreases in accuracy for four different state-of-the-art parsers when applied to social media text. The application of language technology to so- cial media is potentially transformative, leveraging the knowledge and perspectives of millions of peo- ple. But to deliver on this potential, the problems at the core of the NLP pipeline must be addressed. A growing thread of research takes up this chal- lenge, including a shared task and workshop on “parsing the web,” with new corpora which appear to sit somewhere between the Wall Street Journal and Twitter on the spectrum of bad language (Petrov and McDonald, 2012). But perhaps surprisingly, very little of this research has considered why social media language is so different. This review paper attempts to shed some light on this question, sur- veying a strong tradition of empirical and …

Content truncated to 500 words