Paraphrasing 4 Microblog Normalization

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 73–84, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Paraphrasing 4 Microblog Normalization Wang Ling Chris Dyer Alan W Black Isabel Trancoso L2F Spoken Systems Lab, INESC-ID, Lisbon, Portugal Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA Instituto Superior T´ecnico, Lisbon, Portugal {lingwang,cdyer,awb}@cs.cmu.edu isabel.trancoso@inesc-id.pt Abstract Compared to the edited genres that have played a central role in NLP research, mi- croblog texts use a more informal register with nonstandard lexical items, abbreviations, and free orthographic variation. When confronted with such input, conventional text analysis tools often perform poorly. Normalization — replacing orthographically or lexically id- iosyncratic forms with more standard variants — can improve performance. We propose a method for learning normalization rules from machine translations of a parallel corpus of microblog messages. To validate the utility of our approach, we evaluate extrinsically, show- ing that normalizing English tweets and then translating improves translation quality (com- pared to translating unnormalized text) using three standard web translation services as well as a phrase-based translation system trained on parallel microblog data. 1 Introduction Microblogs such as Twitter, Sina Weibo (a popular Chinese microblog service) and Facebook have re- ceived increasing attention in diverse research com- munities (Han and Baldwin, 2011; Hawn, 2009, in- ter alia). In contrast to traditional text domains that use carefully controlled, standardized language, mi- croblog content is often informal, with less adher- ence to conventions regarding punctuation, spelling, and style, and with a higher proportion of dialect or pronouciation-derived orthography. While this diversity itself is an important resource for study- ing, e.g., sociolinguistic variation (Eisenstein et al., 2011; Eisenstein, 2013), it poses challenges to NLP applications developed for more formal domains. If retaining variation due to sociolinguistic or phono- logical factors is not crucial, text normalization can improve performance on downstream tasks (§2). This paper introduces a data-driven approach to learning normalization rules by conceiving of nor- malization as a kind of paraphrasing and taking inspiration from the bilingual pivot approach to paraphrase detection (Bannard and Callison-Burch, 2005) and the observation that translation is an inherently “simplifying” process (Laviosa, 1998; Volansky et al., 2013). Starting from a parallel cor- pus of microblog messages consisting of English paired with several other languages (Ling et al., 2013), we use standard web machine translation sys- tems to re-translate the non-English segment, pro- ducing ⟨English original, English MT⟩ pairs (§3). These are our normalization examples, with MT out- put playing the role of normalized English. Sev- eral techniques for identifying high-precision nor- malization rules are proposed, and we introduce a character-based normalization model to account for predictable character-level processes, like repetition and substitution (§4). We then describe our decod- ing procedure (§5) and show that our normaliza- tion model improve translation quality for English– Chinese microblog translation (§6).1 2 Why Normalize? Consider the English tweet shown in the first row of Table 1 which contains several elements that NLP 1The datasets used in this paper …

Content truncated to 500 words