How Much Data Do You Need? About the Creation of a Ground Truth for Black Letter and the Effectiveness of Neural OCR

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3551–3559 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3551 How Much Data Do You Need? About the Creation of a Ground Truth for Black Letter and the Effectiveness of Neural OCR Phillip Benjamin Str¨obel, Simon Clematide, Martin Volk University of Zurich {pstroebel,siclemat,volk}@cl.uzh.ch Abstract Recent advances in Optical Character Recognition (OCR) and Handwritten Text Recognition (HTR) have led to more accurate text recognition of historical documents. The Digital Humanities heavily profit from these developments, but they still struggle when choosing from the plethora of OCR systems available on the one hand and when defining workflows for their projects on the other hand. In this work, we present our approach to build a ground truth for a historical German-language newspaper published in black letter. We also report how we used it to systematically evaluate the performance of different OCR engines. Additionally, we used this ground truth to make an informed estimate as to how much data is necessary to achieve high-quality OCR results. The outcomes of our experiments show that HTR architectures can successfully recognise black letter text and that a ground truth size of 50 newspaper pages suffices to achieve good OCR accuracy. Moreover, our models perform equally well on data they have not seen during training, which means that additional manual correction for diverging data is superfluous. Keywords: OCR, neural networks, ground truth, Digital Humanities, black letter, German-language, newspapers 1. Introduction For non-digital-born textual data, we need Optical Character Recognition (OCR) to extract the text from scanned images. Although OCR systems have improved in recent years, they still result in a certain error rate. It is first and foremost texts in black letter fonts (also called Gothic fonts) which suffer from limited recognition accuracy. The quality of the OCRed material depends on several factors (we provide examples in Figure 1): • Low distinctiveness of characters, e.g., the “long s” and “f” in older German texts. • Change over time regarding vocabulary and spelling, e.g., Commando (en. command) versus Kommando, both of which have the same meaning, but come in different forms, thereby enlarging the diversity of possible character sequences. • Use of small font sizes (and changes in the fonts themselves). • Mixing of Antiqua and black letter on the same page. • Distortions of the image due to digitisation from bound newspaper collections or imprecise digitisation workflows. • Poor paper quality (e.g., smears, smudges, sometimes due to conservation issues, text from the backside shining through, etc.). Historical documents which have undergone digitisation a few years ago are likely to suffer from low-quality OCR. The possibility of OCR systems to return a confidence score might give an idea about the quality of the extracted text, but such scores are not always available, and even if they are, they are often unreliable. Additionally, libraries and archives often downscale scanned images when distributing them over the web. For the Digital Humanities, this means …

Content truncated to 500 words