Benchmarking SMT Performance for Farsi Using the TEP++ Corpus

Benchmarking SMT Performance for Farsi Using the TEP++ Corpus Peyman Passban, Andy Way, Qun Liu ADAPT Centre School of Computing Dublin City University Dublin, Ireland {ppassban,away,qliu}@computing.dcu.ie Abstract Statistical machine translation (SMT) suf- fers from various problems which are ex- acerbated where training data is in short supply. In this paper we address the data sparsity problem in the Farsi (Persian) lan- guage and introduce a new parallel cor- pus, TEP++. Compared to previous re- sults the new dataset is more efficient for Farsi SMT engines and yields better out- put. In our experiments using TEP++ as bilingual training data and BLEU as a met- ric, we achieved improvements of +11.17 (60%) and +7.76 (63.92%) in the Farsi– English and English–Farsi directions, re- spectively. Furthermore we describe an engine (SF2FF) to translate between for- mal and informal Farsi which in terms of syntax and terminology can be seen as different languages. The SF2FF engine also works as an intelligent normalizer for Farsi texts. To demonstrate its use, SF2FF was used to clean the IWSLT–2013 dataset to produce normalized data, which gave improvements in translation quality over FBK’s Farsi engine when used as training data. 1 Introduction In SMT (Koehn et al., 2003), where the bilingual knowledge comes from parallel corpora, having large datasets is crucial. This issue is compounded when working with low-resource languages, such as Farsi. The poor performance of existing systems © 2015 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. for the Farsi–English pair confirms the necessity of developing a large and representative dataset. Clearly all the existing problems do not originate solely from the data, but not having a reliable train- ing set prevents us from investigating Farsi SMT to the best extent possible. Generating datasets is a time-consuming and expensive process, especially for SMT, in which massive amount of aligned bilingual sentences are required. Accordingly instead of starting from scratch we enriched and refined the existing corpus TEP (Pilevar et al., 2011).1 Despite having a larger alternative (the Mizan2 corpus), TEP was selected as the basis of our work that we clarify further in Section 3 and 4.1. TEP is a collection of film subti- tles in spoken/informal Farsi (SF) that have distinct structures from formal/journalistic Farsi (FF). Ac- cordingly, training an MT engine using this type of data might provide unsatisfactory results when working with FF which is the dominant language of Farsi texts. For this reason TEP was firstly re- fined both manually and automatically, which Sec- tion 3 explains in detail. TEP++ is the refined ver- sion of TEP that is much closer to FF and con- siderably cleaner. Using both TEP and TEP++ we trained several engines for bidirectional translation of the Farsi–English pair, as well as an engine to translate between FF and SF (SF2FF). The next sections explain the challenges of dealing with SF and describe the data preparation process in detail. The structure of paper is as follows. Section …

Content truncated to 500 words