The Unreasonable Effectiveness of Word Representations for Twitter Named Entity Recognition

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 735–745, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics The Unreasonable Effectiveness of Word Representations for Twitter Named Entity Recognition Colin Cherry and Hongyu Guo National Research Council Canada first.last@nrc-cnrc.gc.ca Abstract Named entity recognition (NER) systems trained on newswire perform very badly when tested on Twitter. Signals that were reliable in copy-edited text disappear almost entirely in Twitter’s informal chatter, requiring the con- struction of specialized models. Using well- understood techniques, we set out to improve Twitter NER performance when given a small set of annotated training tweets. To lever- age unlabeled tweets, we build Brown clus- ters and word vectors, enabling generaliza- tions across distributionally similar words. To leverage annotated newswire data, we employ an importance weighting scheme. Taken all together, we establish a new state-of-the-art on two common test sets. Though it is well- known that word representations are useful for NER, supporting experiments have thus far fo- cused on newswire data. We emphasize the ef- fectiveness of representations on Twitter NER, and demonstrate that their inclusion can im- prove performance by up to 20 F1. 1 Introduction Named entity recognition (NER) is the task of find- ing rigid designators as they appear in free text and classifying them into coarse categories such as per- son or location (Nadeau and Sekine, 2007). NER enables many other information extraction tasks such as relation extraction (Bunescu and Mooney, 2005) and entity linking (Ratinov et al., 2011). There is considerable excitement at the prospect of porting information extraction technology to so- cial media platforms such as Twitter. Social media reacts to world events faster than traditional news sources, and its sub-communities pay close attention to topics that other sources might ignore. An early example of the potential inherent in social infor- mation extraction is the Twitter Calendar (Ritter et al., 2012), which detects upcoming events (concerts, elections, video game releases, etc.) based on the anticipatory chatter of Twitter users. Unfortunately, processing social media text presents a unique set of challenges, especially for technologies designed for newswire: Twitter posts are short, the language is informal, capitalization is inconsistent at best, and spelling variations and abbreviations run rampant. Armed with an affordable training set of 1,000 annotated tweets, we establish a strong baseline for Twitter NER using well-understood techniques. We build two unsupervised word representations in or- der to leverage a large collection of unannotated tweets, while a data-weighting technique allows us to benefit from annotated newswire data. Taken together, these two simple ideas establish a new state-of-the-art for both our test sets. We rigorously test the impact of both continuous and cluster-based word representations on Twitter NER, emphasizing the dramatic improvement that they bring. We also bring the experimental methodology of the domain adaptation community to Twitter NER, testing in- domain, out-of-domain and combined training sce- narios, and revealing that it is not trivial to benefit from out-of-domain training data. Finally, …

Content truncated to 500 words