The CMU TransTac 2007 eyes-free two-way speech-to-speech translation system

The CMU TransTac 2007 Eyes-free and Hands-free Two-way Speech-to-Speech Translation System Nguyen Bach, Matthias Eck, Paisarn Charoenpornsawat, Thilo Köhler, Sebastian Stüker, ThuyLinh Nguyen, Roger Hsiao, Alex Waibel, Stephan Vogel, Tanja Schultz, Alan W Black InterACT, Language Technologies Institute, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213, USA. awb@cs.cmu.edu Abstract The paper describes our portable two-way speech-to- speech translation system using a completely eyes- free/hands-free user interface. This system translates between the language pair English and Iraqi Arabic as well as between English and Farsi, and was built within the framework of the DARPA TransTac program. The Farsi language support was developed within a 90-day period, testing our ability to rapidly support new languages. The paper gives an overview of the system’s components along with the individual component objective measures and a discussion of issues relevant for the overall usage of the system. We found that usability, flexibility, and robustness serve as severe constraints on system architecture and design. 1. Introduction In our continuing efforts to construct practical two-way speech-to-speech translation systems, we have developed a generic hands- and eyes-free portable speech translation system which allows an English speaker to converse with a target language speaker. Our systems have been evaluated on a regular basis as part of the DARPA TransTac program. These evaluations are run by NIST, and involve military users and target language users who have never used our system before. The evaluations consist of communicating through the translation device for a number of pre-designed scenarios (which were previously unknown to us). The tests take place both indoors and outdoors. Other systems in the TransTac program include those developed by BBN [5], IBM [6], SRI, Sehda/Fluential, and USC [7] [8]. 2. Challenges The two target languages in the TransTac program are Iraqi Arabic and Farsi. Iraqi Arabic is defined as the spoken form of Arabic used by the people of Iraq in everyday conversations. It is distinct from the formal Modern Standard Arabic (MSA) used in written communication. As Iraqi Arabic is normally not written, even with transcription conventions there is greater variability in the spelling conventions than in a standard written language. Farsi (Persian), mainly spoken in Iran and areas of Afghanistan, also uses the Arabic script, though is not a Semitic language. In general, the target domain is “force protection” which includes checkpoints and house-hold searches, and extends to civil affairs, medical, and training. DARPA has collected a number of spoken dialogs within these domains that have been provided to the participants of the TransTac program. Spoken language translation for the languages Iraqi and Farsi is challenging due to a number of reasons, such as the availability of larger amounts of suitable speech and text data, the lack of extensive language and cultural expertise, language peculiarities in large, and the lack of language convention [3]. Two-way translation between rich inflectional morphology languages, such as Iraqi and Farsi on the one side, and languages with a relatively poor inflectional morphology such as English on the other …

Content truncated to 500 words