Machine translation for information access across the language barrier: the MuST system

MT Summit VII Sept. 1999 Machine Translation for Information Access across the Language Barrier: the MuST System Chin-Yew Lin Information Sciences Institute University of Southern California Marina del Rey, CA 90292-6695, USA Tel: 1-310-822-1511 E-mail: cyl@isi.edu Abstract In this paper we describe the design and implementation of MuST, a multilingual information retrieval, summarization, and translation system. MuST integrates machine translation and other text processing services to enable users to perform cross-language information retrieval using available search services such as commercial Internet search engines. To handle non-standard languages, a new Internet indexing agent can be deployed, specialized local search services can be built, and shallow MT can be added to provide useful functionality. A case study of augmenting MuST with Indonesian is included. MuST adopts ubiquitous web browsers as its primary user interface, and provides tightly integrated automated shallow translation and user biased summarization to help users quickly judge the relevance of documents. 1 Introduction In the past, machine translation systems were most often used in standalone mode, coupled at most with text processing systems. But in the Internet world we live in, there is an increasing need to couple MT engines to other text processing services such as text summarization, information retrieval, and web access. In particular, with the increasing amount of online information and the rapid growth of the number of non-English speaking Internet hosts, it is becoming increasingly important to offer users universal access to valuable information resources in difference languages. The European Multilingual Information Retrieval (EMIR) project [5], the MULINEX project [4], the TwentyOne Project [9], and the cross-language retrieval track in the TREC conference [8] all reflect people's interest in providing interoperability among different language processing environments and multilingual information retrieval. What needs to be done to link MT and IR to create multilingual information retrieval (MLIR)? The problem of language encoding and display used to be an issue, but is now less daunting with the advent of Unicode and web browsers such as Microsoft Internet Explorer and Netscape Navigator. This allows us to focus on the two difficult problems of query translation and result translation. One way to tackle the multilingual information retrieval problem is to translate all the target language text into source language text and then perform monolingual search on the translated text. Oard [20] reports that machine translation (MT) based document translation outperforms MT based query translation. However, translation of 251.840 documents from German to English takes about 10 machine-months on a mix of SPARC 20, SPARC 5. and Ultra SPARC 1 using the Logos translation engine1. Without better machines and high speed/quality MT. we can rule out the practical application of this approach for the web. The document translation approach is even more impractical when fully multilingual information systems were considered, because document translation has to be conducted on each language pair in such a system. We therefore adopt the query translation approach. To translate user queries from source languages to target languages, we need multilingual/bilingual transfer dictionaries or corpora (parallel …

Content truncated to 500 words