The Effect of Translation Quality in MT-Based Cross-Language Information Retrieval

Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 593–600, Sydney, July 2006. c⃝2006 Association for Computational Linguistics The Effect of Translation Quality in MT-Based Cross-Language Information Retrieval Jiang Zhu Haifeng Wang Toshiba (China) Research and Development Center 5/F., Tower W2, Oriental Plaza, No.1, East Chang An Ave., Dong Cheng District Beijing, 100738, China {zhujiang, wanghaifeng}@rdc.toshiba.com.cn Abstract This paper explores the relationship be- tween the translation quality and the re- trieval effectiveness in Machine Transla- tion (MT) based Cross-Language Infor- mation Retrieval (CLIR). To obtain MT systems of different translation quality, we degrade a rule-based MT system by decreasing the size of the rule base and the size of the dictionary. We use the de- graded MT systems to translate queries and submit the translated queries of vary- ing quality to the IR system. Retrieval ef- fectiveness is found to correlate highly with the translation quality of the queries. We further analyze the factors that affect the retrieval effectiveness. Title queries are found to be preferred in MT-based CLIR. In addition, dictionary-based deg- radation is shown to have stronger impact than rule-based degradation in MT-based CLIR. 1 Introduction Cross-Language Information Retrieval (CLIR) enables users to construct queries in one lan- guage and search the documents in another lan- guage. CLIR requires that either the queries or the documents be translated from a language into another, using available translation resources. Previous studies have concentrated on query translation because it is computationally less ex- pensive than document translation, which re- quires a lot of processing time and storage costs (Hull & Grefenstette, 1996). There are three kinds of methods to perform query translation, namely Machine Translation (MT) based methods, dictionary-based methods and corpus-based methods. Corresponding to these methods, three types of translation re- sources are required: MT systems, bilingual wordlists and parallel or comparable corpora. CLIR effectiveness depends on both the design of the retrieval system and the quality of the translation resources that are used. In this paper, we explore the relationship be- tween the translation quality of the MT system and the retrieval effectiveness. The MT system involved in this research is a rule-based English- to-Chinese MT (ECMT) system. We degrade the MT system in two ways. One is to degrade the rule base of the system by progressively remov- ing rules from it. The other is to degrade the dic- tionary by gradually removing word entries from it. In both methods, we observe successive changes on translation quality of the MT system. We conduct query translation with the degraded MT systems and obtain translated queries of varying quality. Then we submit the translated queries to the IR system and evaluate the per- formance. Retrieval effectiveness is found to be strongly influenced by the translation quality of the queries. We further analyze the factors that affect the retrieval effectiveness. Title queries are found to be preferred in MT-based query transla- tion. In addition, the size of the dictionary is shown to have …

Content truncated to 500 words