Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters

Proceedings of NAACL-HLT 2013, pages 380–390, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters Olutobi Owoputi∗ Brendan O’Connor∗ Chris Dyer∗ Kevin Gimpel† Nathan Schneider∗ Noah A. Smith∗ ∗School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA †Toyota Technological Institute at Chicago, Chicago, IL 60637, USA Corresponding author: brenocon@cs.cmu.edu Abstract We consider the problem of part-of-speech tagging for informal, online conversational text. We systematically evaluate the use of large-scale unsupervised word clustering and new lexical features to improve tagging accuracy. With these features, our system achieves state-of-the-art tagging results on both Twitter and IRC POS tagging tasks; Twitter tagging is improved from 90% to 93% accuracy (more than 3% absolute). Quali- tative analysis of these word clusters yields insights about NLP and linguistic phenomena in this genre. Additionally, we contribute the first POS annotation guidelines for such text and release a new dataset of English language tweets annotated using these guidelines. Tagging software, annotation guidelines, and large-scale word clusters are available at: http://www.ark.cs.cmu.edu/TweetNLP This paper describes release 0.3 of the “CMU Twitter Part-of-Speech Tagger” and annotated data. 1 Introduction Online conversational text, typified by microblogs, chat, and text messages,1 is a challenge for natu- ral language processing. Unlike the highly edited genres that conventional NLP tools have been de- veloped for, conversational text contains many non- standard lexical items and syntactic patterns. These are the result of unintentional errors, dialectal varia- tion, conversational ellipsis, topic diversity, and cre- ative use of language and orthography (Eisenstein, 2013). An example is shown in Fig. 1. As a re- sult of this widespread variation, standard model- ing assumptions that depend on lexical, syntactic, and orthographic regularity are inappropriate. There 1Also referred to as computer-mediated communication. ikr ! smh G he O asked V fir P yo D last A name N so P he O can V add V u O on P fb ∧ lololol ! Figure 1: Automatically tagged tweet showing nonstan- dard orthography, capitalization, and abbreviation. Ignor- ing the interjections and abbreviations, it glosses as He asked for your last name so he can add you on Facebook. The tagset is defined in Appendix A. Refer to Fig. 2 for word clusters corresponding to some of these words. is preliminary work on social media part-of-speech (POS) tagging (Gimpel et al., 2011), named entity recognition (Ritter et al., 2011; Liu et al., 2011), and parsing (Foster et al., 2011), but accuracy rates are still significantly lower than traditional well-edited genres like newswire. Even web text parsing, which is a comparatively easier genre than social media, lags behind newspaper text (Petrov and McDonald, 2012), as does speech transcript parsing (McClosky et al., 2010). To tackle the challenge of novel words and con- structions, we create a new Twitter part-of-speech tagger—building on previous work by Gimpel et al. (2011)—that includes new large-scale distribu- tional features. This leads to state-of-the-art results in POS tagging for both Twitter and Internet Relay Chat …

Content truncated to 500 words