From the Paft to the Fiiture: a Fully Automatic NMT and Word Embeddings Method for OCR Post-Correction

Proceedings of Recent Advances in Natural Language Processing, pages 431–436, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_051 431 From the Paft to the Fiiture: a Fully Automatic NMT and Word Embeddings Method for OCR Post-Correction Mika Hämäläinen♠ Simon Hengchen♦ ♠ Department of Digital Humanities, University of Helsinki ♦ COMHIS, University of Helsinki firstname.lastname@helsinki.fi Abstract A great deal of historical corpora suffer from errors introduced by the OCR (optical charac- ter recognition) methods used in the digitiza- tion process. Correcting these errors manually is a time-consuming process and a great part of the automatic approaches have been relying on rules or supervised machine learning. We present a fully automatic unsupervised way of extracting parallel data for training a character- based sequence-to-sequence NMT (neural ma- chine translation) model to conduct OCR error correction. 1 Introduction Historical corpora are a key resource to study so- cial phenomena such as language change in a di- achronic perspective. Approaching this from a computational point of view is especially chal- lenging as historical data tends to be noisy. The noise can come from OCR (optical character recognition) errors, or from the fact that the spelling conventions have changed as the time has passed, as thoroughly described by Piotrowski (2012). However, depending on the NLP or DH task be- ing modelled, some methods can cope with the noise in the data. Indeed, Hill and Hengchen (2019) use a subset of an 18th-century corpus, ECCO,1 and its ground truth version, ECCO- TCP,2 to compare the output of different common DH methods such as authorship attribution, count- based vector space models, and topic modelling, 1Eighteenth Century Collections Online (ECCO) is a dataset which “contains over 180,000 ti- tles (200,000 volumes) and more than 32 million pages”, according to its copyright holder Gale: https://www.gale.com/primary-sources/ eighteenth-century-collections-online. 2ECCO-TCP (Text Creation Partnership) “is a keyed sub- set of ECCO, compiled with the support of 35 libraries and made up of 2,231 documents”. (Hill and Hengchen, 2019) and report that those analyses produce statistically similar output despite noisiness due to OCR. Their conclusion is similar to Rodriquez et al. (2012) in the case of NER and to Franzini et al. (2018) in the case of authorship attribution, but different from Mutuvi et al. (2018) who, specifically on topic modelling for historical newspapers, confirm the often repeated trope of data too dirty to use. How- ever, reducing the noise of OCRed text by apply- ing a post-correction method makes it possible to gain the full potential of the data without having to re-OCR it and opens up the possibility to pro- cess it with the myriad of more precise NLP tools designed for OCR-error free text. This paper focuses on correcting the OCR er- rors in ECCO. We present an unsupervised method based on the advances neural machine transla- tion (NMT) in historical text normalization3. As NMT requires a parallel dataset of OCR errors and their corresponding correct spellings, we propose a method based on word embeddings, a lemma list, and a modern lemmatizer to automatically …

Content truncated to 500 words