An Unsupervised Model of Orthographic Variation for Historical Document Transcription

Proceedings of NAACL-HLT 2016, pages 467–472, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics An Unsupervised Model of Orthographic Variation for Historical Document Transcription Dan Garrette Computer Science & Engineering University of Washington dhg@cs.washington.edu Hannah Alpert-Abrams Comparative Literature Program University of Texas at Austin halperta@gmail.com Abstract Historical documents frequently exhibit ex- tensive orthographic variation, including ar- chaic spellings and obsolete shorthand. OCR tools typically seek to produce so-called diplo- matic transcriptions that preserve these vari- ants, but many end tasks require transcriptions with normalized orthography. In this paper, we present a novel joint transcription model that learns, unsupervised, a probabilistic map- ping between modern orthography and that used in the document. Our system thus pro- duces dual diplomatic and normalized tran- scriptions simultaneously, and achieves a 35% relative error reduction over a state-of-the-art OCR model on diplomatic transcription, and a 46% reduction on normalized transcription. 1 Introduction Optical Character Recognition (OCR) for historical texts, a challenging problem due to unknown fonts and deteriorating documents, is made even more dif- ficult by the fact that orthographic conventions in- cluding spelling, accent usage, and shorthands have not been consistent across the history of printing. For this reason, modern language models (LMs) yield poor performance when trying to recognize characters on the pages of these documents. Further- more, transcription of the actual printed characters may not always be the most desirable output. Greg (1950) describes two types of transcription: one that preserves variants and typographical er- rors, and another that records the substantive con- tent, with this noise removed. Though in 1950 the substantive version was the norm, today these have become two distinct but equally valid tasks. Diplo- matic transcription, the standard in contemporary OCR, preserves the variants of the document valu- able to book historians and linguists. Normalized or modernized transcription recovers the substantive content, producing a text that adheres to modern standards. Normalized transcriptions are easier for users to read, and make large collections of histori- cal texts indexable and searchable (Driscoll, 2006). The current ideal for digital editions of histori- cal texts has been described as a combination of diplomatic and normalized transcription (Pierazzo, 2014). This is generally achieved with a pipeline: first OCR is used to transcribe the document, then an (often manual) post-hoc normalization is performed. However, such a pipeline will result in cascading er- rors from OCR mistakes, and fails to make use of knowledge about modern language during the initial transcription. Additionally, post-processing tools are typically cumbersome language-specific, hand- built systems (Baron and Rayson, 2008; Burns, 2013; Hendrickx and Marquilhas, 2011). In this work, we introduce a novel OCR model designed to jointly produce both diplomatic and nor- malized transcriptions. The model is an extension of Berg-Kirkpatrick et al.’s (2013) Ocular, the state of the art in historical OCR. Ocular’s innovative abil- ity to handle the material challenges of OCR (un- known fonts, uneven inking, etc.) depends on its use of a character n-gram LM. Our model improves the quality of Ocular’s …

Content truncated to 500 words