Training & Quality Assessment of an Optical Character Recognition Model for Northern Haida

Training & Quality Assessment of an Optical Character Recognition Model for Northern Haida Isabell Hubert, Antti Arppe, Jordan Lachler, Eddie A. Santos University of Alberta Edmonton, Alberta, Canada T6G 2E7 isabell.hubert@ualberta.ca, antti.arppe@ualberta.ca, lachler@ualberta.ca, easantos@ualberta.ca Abstract In this paper, we are presenting our work on the creation of the first optical character recognition (OCR) model for Northern Haida, also known as Masset or Xaad Kil, a nearly extinct First Nations language spoken in the Haida Gwaii archipelago in British Columbia, Canada. We are addressing the challenges of training an OCR model for a language with an extensive, non-standard Latin character set as follows: (1) We have compared various training approaches and present the results of practical analyses to maximize recognition accuracy and minimize manual labor. An approach using just one or two pages of Source Images directly performed better than the Image Generation approach, and better than models based on three or more pages. Analyses also suggest that a character’s frequency is directly correlated with its recognition accuracy. (2) We present an overview of current OCR accuracy analysis tools available. (3) We have ported the once de-facto standardized OCR accuracy tools to be able to cope with Unicode input. We hope that our work can encourage further OCR endeavors for other endangered and/or underresearched languages. Our work adds to a growing body of research on OCR for particularly challenging character sets, and contributes to creating the largest electronic corpus for this severely endangered language. Keywords: Optical Character Recognition, Endangered Languages, Indigenous Languages 1. Textual Source This project uses John R. Swanton’s pre-phonemic tran- scriptions of Northern Haida stories from the early 20th century (Swanton, 1908) as a basis, which – at more than 100,000 words and more than half a million charac- ters – form the largest written corpus of Northern Haida in existence and are of excellent print quality: The established English OCR model for the Tesseract engine operated at a character recognition accuracy of 99.44% and a word recognition accuracy of 98.56%, close to (or above) accu- racy levels reported for English in the literature (see Table 1 in Section 4.2.) Good image quality is crucial for any OCR effort as the “[a]ccuracy of single engines largely depends on the training sets created for each collection” (Boschetti et al., 2009, p.164), and as OCR engines “are known to be sensitive to the quality of the document images, with sig- nificant errors observed for even moderately degraded doc- uments” (Dutta et al., 2012, p.1). With native speakers only being found among the el- derly (n.a., 2015a), and the language being far from fully described or analyzed, Swanton’s books provide an excel- lent opportunity in language preservation, revitalization, and technological development efforts (cf. Section 5.) At the same time, our textual source is also of great interest to OCR research, where languages using special character sets still present challenges and where error rates are generally higher for old documents. 2. Comparison of Training Approaches We have chosen to develop the Northern …

Content truncated to 500 words