PJIIT’s systems for WMT 2017 Conference

Proceedings of the Conference on Machine Translation (WMT), Volume 2: Shared Task Papers, pages 416–421 Copenhagen, Denmark, September 711, 2017. c⃝2017 Association for Computational Linguistics PJIIT’s systems for WMT 2017 Conference Krzysztof Wołk Multimedia Department Polish-Japanese Academy of Information Technology, Koszykowa 86, kwolk@pja.edu.pl Krzysztof Marasek Multimedia Department Polish-Japanese Academy of Information Technology, Koszykowa 86, kmarasek@pja.edu.pl Abstract In this paper, we attempt to improve Statistical Machine Translation (SMT) systems between Czech, Latvian and English in WNT’17 News translation task. We also participated in the Biomedical task and produces translation engines from English into Polish, Czech, German, Spanish, French, Hungarian, Romanian and Swedish. To accomplish this, we performed translation model training, created adaptations of training settings for each language pair, and implemented BPE (subword units) for our SMT systems. Innovative tools and data adaptation techniques were employed. Only the official parallel text corpora and monolingual models for the WMT 2017 evaluation campaign were used to train language models, and to develop, tune, and test the system. We explored the use of domain adaptation techniques, symmetrized word alignment models, the unsupervised transliteration models and the KenLM language modeling tool. To evaluate the effects of different preparations on translation results, we conducted experiments and used the BLEU, NIST and TER metrics. Our results indicate that our approach produced a positive impact on SMT quality. 1 Introduction Statistical Machine Translation (SMT) must deal with a number of problems to achieve high quality. These problems include the need to align parallel texts in language pairs and cleaning harvested parallel corpora to remove errors. This is especially true for real-world corpora developed from text harvested from the vast data available on the Internet. Out-Of-Vocabulary (OOV) words must also be handled, as they are inevitable in real-world texts (Wolk and Marasek, 2014a). The lack of enough parallel corpora for some less popular languages is another significant challenge for SMT. Since the approach is statistical in nature, a significant amount of quality language pair data is needed to improve translation accuracy. In addition, very general translation systems that work in a general text domain have accuracy problems in specific domains. SMT systems are more accurate on corpora from a domain that is not too wide. This exacerbates the data problem, calling for the enhancement of parallel corpora for particular text domains (Wolk and Marasek, 2014b). This paper describes SMT research that addresses these problems, particularly domain adaptation within the limits of permissible data for the WMT 2017 campaign. To accomplish this, we performed model training, created adaptations of training settings and data for each language pair. Innovative tools and data adaptation techniques were employed. We explored the use of domain adaptation techniques, symmetrized word alignment models, the unsupervised transliteration models, and the KenLM language modeling tool (Heafield, 2011). To evaluate the effects of different preparations on translation results, we conducted experiments and evaluated the results using standard SMT metrics (Koehn et al., 2007). The languages translated during this research were: Czech, Latvian and English in WNT’17 News translation task. …

Content truncated to 500 words