Evaluating the Translation Accuracy of a Novel Language-Independent MT Methodology

Proceedings of COLING 2012: Technical Papers, pages 2569–2584, COLING 2012, Mumbai, December 2012. Evaluating the translation accuracy of a novel language-independent MT methodology George TAMBOURATZIS, Sokratis SOFIANOPOULOS, Marina VASSILIOU Institute for Language & Speech Processing, Athena R.C. 6 Artemidos & Epidavrou Str., Paradissos Amaroussiou, 151 25 Athens, Greece giorg_t@ilsp.gr,s_sofian@ilsp.gr,mvas@ilsp.gr ABSTRACT The current paper evaluates the performance of the PRESEMT methodology, which facilitates the creation of machine translation (MT) systems for different language pairs. This methodology aims to develop a hybrid MT system that extracts translation information from large, predominantly monolingual corpora, using pattern recognition techniques. PRESEMT has been designed to have the lowest possible requirements on specialised resources and tools, given that for many languages (especially less widely used ones) only limited linguistic resources are available. In PRESEMT, the main translation process is divided into two phases, the first determining the overall structure of a target language (TL) sentence, and the second disambiguating between alternative translations for words or phrases and establishing local word order. This paper describes the latest version of the system and evaluates its translation accuracy, while also benchmarking the PRESEMT performance by comparing it with other established MT systems using objective measures. KEYWORDS: hybrid machine translation; language-independent methodology; MT evaluation 2569 1 Introduction The Machine Translation (MT) task has been studied for a number of decades, but still remains to a large extent an issue unresolved, as the performance delivered by the best current systems still falls short of the required quality. Since the number of texts available over the World Wide Web is ever increasing, and these texts may be written in one of several hundred languages, the requirement for automatically performing translation of an acceptable quality remains a prime objective. A number of MT paradigms have been proposed, the main ones including Rule-Based MT (RBMT), Statistical MT (SMT) and Example-Based MT (EBMT). Furthermore, the requirement for covering an ever increasing combination of Source to Target language (SL to TL) combinations necessitates the development of language-independent methodologies. Currently most MT approaches are based on the SMT paradigm (Koehn, 2010). SMT uses dedicated algorithms that do not employ language-specific rules and is thus portable to new language pairs, provided the necessary training data are available. The main SMT constraint is the need for SL-TL bilingual corpora of a sufficient size (of the order of a million parallel sentences) to allow the building of accurate translation models. Such corpora are hard to obtain, particularly when less widely-used languages are involved. Besides, the process of compiling and verifying such corpora is expensive in terms of both manpower and time. In EBMT, translations are generated by analogy, where the system has available a set of known pairs of input sentence (in SL) and corresponding translation (in TL). Then, each new input sentence is broken down to non-overlapping phrases, which are translated using the translation examples as a reference. The translated sentence is finally composed by combining the translated phrases. Another paradigm is hybrid MT, which combines ideas and techniques …

Content truncated to 500 words