Noisy Parallel Data Alignment

Findings of the Association for Computational Linguistics: EACL 2023, pages 1501–1513 May 2-6, 2023 ©2023 Association for Computational Linguistics Noisy Parallel Data Alignment Ruoyu Xie, Antonios Anastasopoulos Department of Computer Science, George Mason University {rxie, antonis}@gmu.edu Abstract An ongoing challenge in current natural lan- guage processing is how its major advance- ments tend to disproportionately favor resource- rich languages, leaving a significant number of under-resourced languages behind. Due to the lack of resources required to train and evaluate models, most modern language technologies are either nonexistent or unreliable to process endangered, local, and non-standardized lan- guages. Optical character recognition (OCR) is often used to convert endangered language doc- uments into machine-readable data. However, such OCR output is typically noisy, and most word alignment models are not built to work under such noisy conditions. In this work, we study the existing word-level alignment mod- els under noisy settings and aim to make them more robust to noisy data. Our noise simula- tion and structural biasing method, tested on multiple language pairs, manages to reduce the alignment error rate on a state-of-the-art neural- based alignment model up to 59.6%.1 1 Introduction Modern optical character recognition (OCR) soft- ware achieves good performance on documents in high-resource standardized languages, produc- ing machine-readable text which can be used for many downstream natural language process- ing (NLP) tasks and various applications (Ignat et al., 2022; Van Strien et al., 2020; Amrhein and Clematide, 2018). However, attaining the same level of quality for texts in less-resourced local and non-standardized languages remains an open problem (Rijhwani et al., 2020). The promise of OCR is particularly appealing for endangered languages, for which material might ex- ist in non-machine-readable formats, such as physi- cal books or educational materials. Digitizing such 1Data and code are available online: https://github. com/ruoyuxie/noisy_parallel_data_alignment Figure 1: Synthetic data example in Griko with charac- ter differences highlighted. Our synthetic data manage to mimic the real OCR noise. material could lead to the creation of NLP tech- nologies for such otherwise severely under-served communities (Bustamante et al., 2020). Beyond the primary goal of digitizing printed material in endangered languages, the need for ro- bust alignment tools is wider. The majority of the world’s languages are being traditionally oral (Bird, 2020), which implies that to obtain textual data at scale one would need to rely on automatic speech recognition (ASR), which in turn would produce invariably noisy outputs. It is worth noting that the availability of translations can significantly im- prove systems beyond machine translation (MT), such as OCR (Rijhwani et al., 2020) or ASR (Anas- tasopoulos and Chiang, 2018). This creates a chicken-and-egg situation: on one hand, OCR and ASR can be used to obtain noisy parallel data; on the other hand, having good quality aligned data can improve OCR or ASR. In this vein, We focus on the scenario of dig- itizing texts in a less-resourced language along with their translations (usually high-resource and/or widely spoken) similar to Rijhwani et al. (2020). Digitizing parallel documents can also …

Content truncated to 500 words