The Value of Monolingual Crowdsourcing in a Real-World Translation Scenario: Simulation using Haitian Creole Emergency SMS Messages

Proceedings of the 6th Workshop on Statistical Machine Translation, pages 399–404, Edinburgh, Scotland, UK, July 30–31, 2011. c⃝2011 Association for Computational Linguistics The Value of Monolingual Crowdsourcing in a Real-World Translation Scenario: Simulation using Haitian Creole Emergency SMS Messages Chang Hu‡, Philip Resnik†‡, Yakov Kronrod† Vladimir Eidelman‡, Olivia Buzek†‡, Benjamin B. Bederson‡ †UMIACS and Department of Linguistics ‡UMIACS and Department of Computer Science University of Maryland, College Park {changhu,bederson}@cs.umd.edu {resnik,vlad,buzek}@umiacs.umd.edu yakov@umd.edu Abstract MonoTrans2 is a translation system that com- bines machine translation (MT) with human computation using two crowds of monolin- gual source (Haitian Creole) and target (En- glish) speakers. We report on its use in the WMT 2011 Haitian Creole to English trans- lation task, showing that MonoTrans2 trans- lated 38% of the sentences well compared to Google Translate’s 25%. 1 Introduction One of the most remarkable success stories to come out of the January 2010 earthquake in Haiti in- volved translation (Munro, 2010). While other forms of emergency response and communication channels were failing, text messages were still get- ting through, so a number of people came together to create a free phone number for emergency text mes- sages, which allowed earthquake victims to report those who were trapped or in need of medical atten- tion. The problem, of course, was that most people were texting in Haitian Creole (Kreyol), a language not many of the emergency responders understood, and few, if any, professional translators were avail- able. The availability of usable translations literally became a matter of life and death. In response to this need, Stanford University grad- uate student Rob Munro coordinated the rapid cre- ation of a crowdsourcing framework, which allowed volunteers – including, for example, Haitian expa- triates and French speakers – to translate messages, providing responders with usable information in as little as ten minutes. Translations may not have been perfect, but to a woman in labor, it had to have made a big difference for English-speaking responders to see Undergoing children delivery Delmas 31 instead of Fanm gen tranche pou f`e yon pitit nan Delmas 31. What about a scenario, though, in which even am- ateur bilingual volunteers are hard to find, or too few in number? What about a scenario, e.g. the March 2011 earthquake and tsunami in Japan, in which there are many people worldwide who wish to help but are not fluent in both the source and tar- get languages? For the last few years, we have been exploring the idea of monolingual crowdsourcing for translation – that is, technology-assisted collaborative transla- tion involving crowds of participants who know only the source or target language (Buzek et al., 2010; Hu, 2009; Hu et al., 2010; Hu et al., 2011; Resnik et al., 2010). Our MonoTrans2 framework has pre- viously shown very promising results on children’s books: on a test set where Google Translate pro- duced correct translations for only 10% of the input sentences, monolingual German and Spanish speak- ers using our framework produced translations that were fully correct …

Content truncated to 500 words