Connectionist and Symbolic Processing in Speech-to-Speech Translation: The JANUS System

[MT Summit III, Washington, DC, USA, July 1991] Connectionist and Symbolic Processing in Speech-to-Speech Translation: The JANUS System A. N. Jain A. E. McNair A. Waibel H. Saito A. G. Hauptmann J. Tebelskis * School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213-3890 Abstract We present JANUS, a speech-to-speech trans- lation system that utilizes diverse process- ing strategies including connectionist learning, traditional AI knowledge representation ap- proaches, dynamic programming, and stochas- tic techniques. JANUS translates continuously spoken English utterances into Japanese and German speech utterances. The overall system performance on a corpus of conference regis- tration conversations is 87%. Two versions of JANUS are compared: one using an LR parser (JANUS-LR) and one using a neural-network based parser (JANUS-NN). Performance re- sults are mixed, with JANUS-LR deriving ben- efit from a tighter language model and JANUS- NN benefiting from greater flexibility. 1 Introduction In an age of increasing internationalization, efficient and rapid communication between people around the world has become a necessity of modern business life. It has created a rising need for better communication tools that could help bridge the language and culture gaps that still isolate one people from another. In this spirit, auto- mated speech translation has become a research target for a number of research laboratories (e.g. [Morimoto et al., 1990]). Real-time translation of telephone con- versations is an ambitious project. It requires the inte- gration of three component technologies: speech recog- nition, machine translation, and speech synthesis. Each of these technologies is currently under active research. In this paper, we present JANUS 1, a speech-to- speech translation system developed at Carnegie Mel- lon. JANUS is able to translate continuously spoken English speech utterances into Japanese and German speech utterances. The system consists of several sub- components that utilize different processing strategies ranging from connectionist systems to LR parsing al- gorithms (see Fig. 1). JANUS currently operates on a *The authors gratefully acknowledge the support of DARPA, the National Science Foundation, ATR Interpret- ing Telephony Research Laboratories, NEC Corporation, and Siemens Corporation. 1 The system is named after the Roman god with two faces. conference registration dialog task. In the conversations, a caller is attempting to obtain information or to regis- ter for an international conference by interacting with a conference secretary. The speech dialogs were acted (read) out under benign recording conditions. As a speech recognition front-end, we use a continuous speech recognition system based on connectionist acous- tic modeling (LPNNs) and stochastic language modeling techniques (bigram grammars). At the language process- ing and translation level, we describe and evaluate two modules using contrasting computational techniques- knowledge-based versus connectionist language process- ing. Speech synthesis, finally, is performed by two com- mercial text-to-speech synthesis devices, one for German and one for Japanese. The system's ability to deliver the correct translation in the face of potential recogni- tion errors is our primary goal. A long-term goal is to demonstrate tolerance to speech effects such as ungram- maticality, stuttering, interjections, etc. Figure 1: JANUS system components, In …

Content truncated to 500 words