How Noisy Social Media Text, How Diffrnt Social Media Sources?

International Joint Conference on Natural Language Processing, pages 356–364, Nagoya, Japan, 14-18 October 2013. How Noisy Social Media Text, How Diffrnt Social Media Sources? Timothy Baldwin,♠♥ Paul Cook,♥ Marco Lui,♠♥ Andrew MacKinlay♠♥ and Li Wang♠♥ ♠ NICTA Victoria Research Laboratory ♥ Department of Computing and Information Systems, The University of Melbourne tb@ldwin.net, paulcook@unimelb.edu.au, mhlui@unimelb.edu.au, Andrew.MacKinlay@nicta.com.au, li.wang.d@gmail.com Abstract While various claims have been made about text in social media text being noisy, there has never been a systematic study to investigate just how linguistically noisy or otherwise it is over a range of social media sources. We explore this question empirically over popular social media text types, in the form of YouTube comments, Twitter posts, web user forum posts, blog posts and Wikipedia, which we compare to a reference corpus of edited English text. We first extract out various descrip- tive statistics from each data type (includ- ing the distribution of languages, average sentence length and proportion of out-of- vocabulary words), and then investigate the proportion of grammatical sentences in each, based on a linguistically-motivated parser. We also investigate the relative similarity between different data types. 1 Introduction Various claims have been made about social me- dia text being “noisy” (Java, 2007; Becker et al., 2009; Yin et al., 2012; Preotiuc-Pietro et al., 2012; Eisenstein, 2013, inter alia). However, there has been little effort to quantify the extent to which social media text is more noisy than conventional, edited text types. Moreover, social media comes in many flavours — including microblogs, blogs, and user-generated comments — and research has tended to focus on a specific data source, such as Twitter or blogs. A natural question to ask is how different the textual content of the myriad of so- cial media types are from one another. This is an important first step towards building a general- purpose suite of social media text processing tools. Most research to date on social media text has used very shallow text processing (such as keyword-based time-series analysis), with natu- ral language processing (NLP) tools such as part- of-speech taggers and parsers tending to be dis- favoured because of the perceived intractability of applying them to social media text. However, there has been little analysis quantifying just how hard it is to apply NLP to social media text, or how intractable the data is for NLP tools. This paper addresses the two issues above. We build corpora from a variety of popular so- cial media sources, including microblogs, user- generated comments, user forums, blogs, and collaboratively-authored content. We then com- pare these corpora to more conventional texts through a variety of statistical and linguistic anal- yses to quantitatively assess the relative extent to which they are “noisy”, and quantify similar- ities between them. Our findings indicate that there are certainly differences between social me- dia sites, but that if we focus our attention on En- glish text, there are striking similarities, and that even sources such as Twitter may be more “NLP- tractable” than they are often …

Content truncated to 500 words