Part-of-Speech Tagging for Twitter with Adversarial Neural Networks

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2411–2420 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Part-of-Speech Tagging for Twitter with Adversarial Neural Networks Tao Gui, Qi Zhang∗, Haoran Huang, Minlong Peng, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {tgui16,qz,huanghr15,mlpeng16,xjhuang}@fudan.edu.cn Abstract In this work, we study the problem of part- of-speech tagging for Tweets. In contrast to newswire articles, Tweets are usually informal and contain numerous out-of- vocabulary words. Moreover, there is a lack of large scale labeled datasets for this domain. To tackle these challenges, we propose a novel neural network to make use of out-of-domain labeled data, unlabeled in-domain data, and labeled in- domain data. Inspired by adversarial neural networks, the proposed method tries to learn common features through adversarial discriminator. In addition, we hypothesize that domain-specific fea- tures of target domain should be preserved in some degree. Hence, the proposed method adopts a sequence-to-sequence au- toencoder to perform this task. Experi- mental results on three different datasets show that our method achieves better per- formance than state-of-the-art methods. 1 Introduction During the last decade, social media have become extremely popular, on which billions of user- generated contents are posted every day. Many users have been writing about their thoughts and lives on the go. The massive unstructured data from social media provides valuable informa- tion for a variety of applications such as stock prediction (Bollen et al., 2011), public health analysis (Wilson and Brownstein, 2009; Paul and Dredze, 2011), real-time event detection (Sakaki et al., 2010), and so on. The quality of these appli- cations is highly impacted by the performance of natural language processing tasks. ∗Corresponding author. @DORSEY33 lol aw i thought u was talkin bout another time . nd i dnt see u either ! USR UH UH PRP VBD PRP VBD VBG IN DT NN . CC PRP VBP VB PRP RB . Figure 1: An example of tagged Tweet, which contains nonstandard orthography, emoticon, and abbreviation. The tagset is defined similar as that of PTB (Marcus et al., 1993). Part-of-speech (POS) tagging is one of the most important natural language processing tasks. has also been widely used in the social mediaIt analysis systems (Ritter et al., 2012; Lamb et al., 2013; Kiritchenko et al., 2014). Most state- of-the-art POS tagging approaches are based on supervised methods. Hence, they usually require a large amount of annotated data to train models. Many datasets have been constructed for POS tagging task. Because newswire articles are carefully edited, benchmarks usually use them for annotation (Marcus et al., 1993). However, user- generated contents on social media are usually informal and contain many nonstandard lexical items. Moreover, the difference in domains be- tween training data and evaluation data may heav- ily impact the performance of approaches based on supervised methods (Caruana and Niculescu- Mizil, 2006). Hence, most POS tagging meth- ods cannot achieve the same performance as reported …

Content truncated to 500 words