Parsing Tweets into Universal Dependencies

Proceedings of NAACL-HLT 2018, pages 965–975 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Parsing Tweets into Universal Dependencies Yijia Liu Harbin Institute of Technology yjliu@ir.hit.edu.cn Yi Zhu University of Cambridge yz568@cam.ac.uk Wanxiang Che Bing Qin Harbin Institute of Technology Nathan Schneider Georgetown University Noah A. Smith University of Washington Abstract We study the problem of analyzing tweets with Universal Dependencies (UD; Nivre et al., 2016). We extend the UD guide- lines to cover special constructions in tweets that affect tokenization, part-of- speech tagging, and labeled dependen- cies. Using the extended guidelines, we create a new tweet treebank for English (TWEEBANK V2) that is four times larger than the (unlabeled) TWEEBANK V1 intro- duced by Kong et al. (2014). We char- acterize the disagreements between our annotators and show that it is challeng- ing to deliver consistent annotation due to ambiguity in understanding and ex- plaining tweets. Nonetheless, using the new treebank, we build a pipeline sys- tem to parse raw tweets into UD. To over- come annotation noise without sacrific- ing computational efficiency, we propose a new method to distill an ensemble of 20 transition-based parsers into a single one. Our parser achieves an improvement of 2.2 in LAS over the un-ensembled baseline and outperforms parsers that are state-of- the-art on other treebanks in both accuracy and speed. 1 Introduction NLP for social media messages is challenging, re- quiring domain adaptation and annotated datasets (e.g., treebanks) for training and evaluation. Pi- oneering work by Foster et al. (2011) annotated 7,630 tokens’ worth of tweets according to the phrase-structure conventions of the Penn Treebank (PTB; Marcus et al., 1993), enabling conversion to Stanford Dependencies. Kong et al. (2014) further studied the challenges in annotating tweets and presented a tweet treebank (TWEEBANK), consist- ing of 12,149 tokens and largely following con- ventions suggested by Schneider et al. (2013), fairly close to Yamada and Matsumoto (2003) de- pendencies (without labels). Both annotation ef- forts were highly influenced by the PTB, whose guidelines have good grammatical coverage on newswire. However, when it comes to informal, unedited, user-generated text, the guidelines may leave many annotation decisions unspecified. Universal Dependencies (Nivre et al., 2016, UD) were introduced to enable consistent anno- tation across different languages. To allow such consistency, UD was designed to be adaptable to different genres (Wang et al., 2017) and languages (Guo et al., 2015; Ammar et al., 2016). We pro- pose that analyzing the syntax of tweets can bene- fit from such adaptability. In this paper, we intro- duce a new English tweet treebank of 55,607 to- kens that follows the UD guidelines, but also con- tends with social media-specific challenges that were not covered by UD guidelines.1 Our anno- tation includes tokenization, part-of-speech (POS) tags, and (labeled) Universal Dependencies. We characterize the disagreements among our annota- tors and find that consistent annotation is still chal- lenging to deliver even with the extended guide- lines. Based on these annotations, we nonetheless de- signed a pipeline to …

Content truncated to 500 words