Twitter Translation using Translation-Based Cross-Lingual Retrieval

Proceedings of the 7th Workshop on Statistical Machine Translation, pages 410–421, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics Twitter Translation using Translation-Based Cross-Lingual Retrieval Laura Jehl and Felix Hieber and Stefan Riezler Department of Computational Linguistics Heidelberg University 69120 Heidelberg, Germany {jehl,hieber,riezler}@cl.uni-heidelberg.de Abstract Microblogging services such as Twitter have become popular media for real-time user- created news reporting. Such communica- tion often happens in parallel in different lan- guages, e.g., microblog posts related to the same events of the Arab spring were written in Arabic and in English. The goal of this paper is to exploit this parallelism in order to eliminate the main bottleneck in automatic Twitter translation, namely the lack of bilin- gual sentence pairs for training SMT systems. We show that translation-based cross-lingual information retrieval can retrieve microblog messages across languages that are similar enough to be used to train a standard phrase- based SMT pipeline. Our method outper- forms other approaches to domain adaptation for SMT such as language model adaptation, meta-parameter tuning, or self-translation. 1 Introduction Among the various social media platforms, mi- croblogging services such as Twitter1 have become popular communication tools. This is due to the easy accessibility of microblogging platforms via inter- net or mobile phones, and due to the need for a fast mode of communication that microblogging satis- fies: Twitter messages are short (limited to 140 char- acters) and simultaneous (due to frequent updates by prolific microbloggers). Twitter users form a social network by “following” the updates of other users, either reciprocal or one-way. The topics discussed in Twitter messages range from private chatter to im- portant real-time witness reports. 1http://twitter.com/ Events such as the Arab spring have shown the power and also the shortcomings of this new mode of communication. Microblogging services played a crucial role in quickly spreading the news about im- portant events, furthermore they were useful in help- ing organizers plan their protest. The fact that news on microblogging platforms is sometimes ahead of newswire is one of the most interesting facets of this new medium. However, while Twitter messag- ing is happening in multiple languages, most net- works of “friends” and “followers” are monolingual and only about 40% of all messages are in English2. One solution to sharing news quickly and interna- tionally was crowdsourcing manual translations, for example at Meedan3, a nonprofit organization built to share news and opinion between the Arabic and English speaking world, by translating articles and blogs, using machine translation and human expert corrections. The goal of our research is to automate this trans- lation process, with a further aim of providing rapid crosslingual data access for downstream applica- tions. The automated translation of microblogging messages is facing two main problems. First, there are no bilingual sentence pair data from microblog- ging domains available. Second, the colloquial, non- standard language of many microblogging messages makes it very difficult to adapt a machine translation system trained on any of the available bilingual re- sources such as transcriptions from …

Content truncated to 500 words