OCR Improves Machine Translation for Low-Resource Languages

Findings of the Association for Computational Linguistics: ACL 2022, pages 1164 - 1174 May 22-27, 2022 c⃝2022 Association for Computational Linguistics OCR Improves Machine Translation for Low-Resource Languages Oana Ignat1 Jean Maillard2 Vishrav Chaudhary3∗ Francisco Guzmán2 University of Michigan1 Meta AI2 Microsoft Turing3 oignat@umich.edu, jean@maillard.it, vchaudhary@microsoft.com fguzman@fb.com Abstract We aim to investigate the performance of cur- rent OCR systems on low resource languages and low resource scripts. We introduce and make publicly available a novel benchmark, OCR4MT, consisting of real and synthetic data, enriched with noise, for 60 low-resource languages in low resource scripts. We evalu- ate state-of-the-art OCR systems on our bench- mark and analyse most common errors. We show that OCR monolingual data is a valu- able resource that can increase performance of Machine Translation models, when used in backtranslation. We then perform an ablation study to investigate how OCR errors impact Machine Translation performance and deter- mine what is the minimum level of OCR qual- ity needed for the monolingual data to be use- ful for Machine Translation. 1 Introduction Despite many recent successes, Machine Transla- tion still lacks support or fails to achieve good per- formance for most low-resource languages, which represent a very large fraction of the languages spo- ken by the world’s population (Fan et al., 2020; Wenzek et al., 2020; Goyal et al., 2021). The poor performance in these settings can largely be attributed to the lack of training data. Many techniques for improving Machine Transla- tion, such as backtranslation (Sennrich et al., 2016; Edunov et al., 2018; Zhang et al., 2020) and ap- proaches which make use of pre-trained language models (Gao et al., 2019; Chen et al., 2021; Liu et al., 2021), rely heavily on high quality monolin- gual data, which is not readily available for low- resource languages. Fortunately, many books and other resources in these languages have been dig- itized and made available online. However, this textual data is “locked” away in formats such as PDFs and images, which are not readily accessible. ∗Work done while at Meta AI. As a result, there are large unexplored collections of data in many languages which could be used as a source for monolingual data. For example, one Nepali books corpus1, contains around 342M tokens, which would potentially make it one of the largest sources of monolingual data for this language. A solution to this problem is to rely on modern Optical Character Recognition (OCR) tools to ex- tract the text. Unfortunately however, most of the OCR models have only been evaluated on a hand- ful of languages, and public benchmarks for low- resource scripts and languages are lacking (Smith, 2007a; Wick et al., 2020). As a result, a compre- hensive evaluation of OCR tools, particularly for low-resource languages and scripts, is still an open problem. Moreover, there is little-to-no understand- ing of the downstream effect that recognition errors will have on the data augmentation techniques that make use of high-quality monolingual data, such as the methods that low-resource language translation typically …

Content truncated to 500 words