Haitian Creole: How to Build and Ship an MT Engine from Scratch in 4 days, 17 hours, & 30 minutes

Haitian Creole: How to Build and Ship an MT Engine from Scratch in 4 days, 17 hours, & 30 minutes William D. Lewis Microsoft Research One Microsoft Way Redmond, WA 98052 wilewis@microsoft.com Abstract We describe the effort of the Microsoft Translator team to develop a Haitian Cre- ole statistical machine translation engine from scratch in a matter of days. Haitian Creole presents a number of difficulties for devleoping an SMT system, princi- pal among these is the lack of significant amounts of parallel training data and an in- consistent orthography, both of which lead to data sparseness. We demonstrate, how- ever, that it is possible to build a trans- lation engine of reasonable quality over very little data by engaging with the native language community and reducing data sparseness in creative ways. As such, we show that MT as a technology and as a ser- vice can be deployed rapidly in crisis situ- ations. 1 Credits Credit is due to all members of the Microsoft Translator team who spent many sleepless nights putting together and shipping the Haitian Creole Translators. I am proud and humbled to call you colleagues. Our whole team in turn is indebted to the researchers and data providers who helped us locate or directly provided us with training data for our systems, to the Butler Hill Group and their employees who donated many, many hours to the effort, to Moravia Worldwide and Welocalize who helped translate content, and most especially to the volunteers involved in the relief effort, especially those at Ushahidi and Mission 4636, who offered crucial advice and provided us with data for our en- c⃝ 2010 European Association for Machine Translation. gine. This entire effort would not have succeeded without your help. 2 Introduction The unprecedented disaster in Haiti triggered a massive relief effort from around the world. Since a significant portion of the population of Haiti speaks Haitian Creole—approximately seven out of every eight people in Haiti speak the lan- guage1— and since Haitian Creole is not widely spoken outside of Haiti, it became obvious that Machine Translation might be a useful tool for the relief effort in Haiti, where it could be applied to translate emergency relief documents, medical documents, SMS text messages, and even common phrases and expressions. Unfortunately, no readily available Machine Translation engine existed for Haitian Creole at the time. Our team received an e-mail from colleagues on the ground in Haiti ask- ing if we could develop a translation engine to aid in the relief effort. Here we relate this effort and the daunting challenges we faced to produce such an engine from scratch in just a few days. 3 Motivation Haitian Creole is a resource poor language spoken principally on the western portion of the Island of Hispaniola. On January 12th, 2010 a devastating earthquake struck the island, with an epicenter in 1This calculation based on the number of speakers in Haiti of 6,960,000 in 2001 as listed Ethnologue (http://www.ethnologue.com/show language.asp?code=hat), and an extrapolation of the …

Content truncated to 500 words