A Beam-Search Decoder for Normalization of Social Media Text with Application to Machine Translation

Proceedings of NAACL-HLT 2013, pages 471–481, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics A Beam-Search Decoder for Normalization of Social Media Text with Application to Machine Translation Pidong Wang Department of Computer Science National University of Singapore 13 Computing Drive Singapore 117417 wangpd@comp.nus.edu.sg Hwee Tou Ng Department of Computer Science National University of Singapore 13 Computing Drive Singapore 117417 nght@comp.nus.edu.sg Abstract Social media texts are written in an infor- mal style, which hinders other natural lan- guage processing (NLP) applications such as machine translation. Text normalization is thus important for processing of social media text. Previous work mostly focused on nor- malizing words by replacing an informal word with its formal form. In this paper, to fur- ther improve other downstream NLP applica- tions, we argue that other normalization oper- ations should also be performed, e.g., missing word recovery and punctuation correction. A novel beam-search decoder is proposed to ef- fectively integrate various normalization oper- ations. Empirical results show that our system obtains statistically significant improvements over two strong baselines in both normaliza- tion and translation tasks, for both Chinese and English. 1 Introduction Social media texts include SMS (Short Message Service) messages, Twitter messages, Facebook up- dates, etc. They are different from formal texts due to their significant informal characteristics, so they always pose difficulties for applications such as ma- chine translation (MT) (Aw et al., 2005) and named entity recognition (Liu et al., 2011), because of a lack of training data containing informal texts. Thus, the applications always suffer from a substantial per- formance drop when evaluated on social media texts. For example, Ritter et al. (2011) reported a drop from 90% to 76% on part-of-speech tagging, and Foster et al. (2011) found a drop of 20% in depen- dency parsing. Creating training data of social media texts specif- ically for a text processing task is time-consuming. For example, to create parallel Chinese-English training texts for translation of social media texts, it takes three minutes on average to translate an in- formally written social media text of eleven words from Chinese into English. On the other hand, it takes thirty seconds to normalize the same message, a six-fold increase in speed. After training a text nor- malization system to normalize social media texts, we can use an existing statistical machine translation (SMT) system trained on normal texts (non-social media texts) to carry out translation. So we argue that normalization followed by regular translation is a more practical approach. Thus, text normalization is important for social media text processing. Most previous work on normalization of social media text focused on word substitution (Beaufort et al., 2010; Gouws et al., 2011; Han and Baldwin, 2011; Liu et al., 2012). However, we argue that some other normalization operations besides word substitution are also critical for subsequent natu- ral language processing (NLP) applications, such as missing word recovery (e.g., zero pronouns) and punctuation correction. In this paper, we propose a novel beam-search decoder for normalization of social media text …

Content truncated to 500 words