Bilingual dictionary generation for low-resourced language pairs

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 862–870, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Bilingual dictionary generation for low-resourced language pairs Varga István Yamagata University, Graduate School of Science and Engineering dyn36150@dip.yz.yamagata-u.ac.jp Yokoyama Shoichi Yamagata University, Graduate School of Science and Engineering yokoyama@yz.yamagata-u.ac.jp Abstract Bilingual dictionaries are vital resources in many areas of natural language processing. Numerous methods of machine translation re- quire bilingual dictionaries with large cover- age, but less-frequent language pairs rarely have any digitalized resources. Since the need for these resources is increasing, but the hu- man resources are scarce for less represented languages, efficient automatized methods are needed. This paper introduces a fully auto- mated, robust pivot language based bilingual dictionary generation method that uses the WordNet of the pivot language to build a new bilingual dictionary. We propose the usage of WordNet in order to increase accuracy; we also introduce a bidirectional selection method with a flexible threshold to maximize recall. Our evaluations showed 79% accuracy and 51% weighted recall, outperforming represen- tative pivot language based methods. A dic- tionary generated with this method will still need manual post-editing, but the improved recall and precision decrease the work of hu- man correctors. 1 Introduction In recent decades automatic and semi-automatic machine translation systems gradually managed to take over costly human tasks. This much wel- comed change can be attributed not only to major developments in techniques regarding translation methods, but also to important translation re- sources, such as monolingual or bilingual dic- tionaries and corpora, thesauri, and so on. How- ever, while widely used language pairs can fully take advantage of state-of-the-art developments in machine translation, certain low-frequency, or less common language pairs lack some or even most of the above mentioned translation re- sources. In that case, the key to a highly accurate machine translation system switches from the choice and adaptation of the translation method to the problem of available translation resources between the chosen languages. One possible solution is bilingual corpus ac- quisition for statistical machine translation (SMT). However, for highly accurate SMT sys- tems large bilingual corpora are required, which are rarely available for less represented lan- guages. Rule or sentence pattern based systems are an attractive alternative, for these systems the need for a bilingual dictionary is essential. Our paper targets bilingual dictionary genera- tion, a resource which can be used within the frameworks of a rule or pattern based machine translation system. Our goal is to provide a low- cost, robust and accurate dictionary generation method. Low cost and robustness are essential in order to be re-implementable with any arbitrary language pair. We also believe that besides high precision, high recall is also crucial in order to facilitate post-editing which has to be performed by human correctors. For improved precision, we propose the usage of WordNet, while for good recall we introduce a bidirectional selection method with local thresholds. Our paper is structured as follows: first we overview the most significant …

Content truncated to 500 words