Arabic OCR Error Correction Using Character Segment Correction, Language Modeling, and Shallow Morphology

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 408–414, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Arabic OCR Error Correction Using Character Segment Correction, Language Modeling, and Shallow Morphology Walid Magdy and Kareem Darwish IBM Technology Development Center P.O. Box 166 El-Ahram, Giza, Egypt {wmagdy,darwishk}@eg.ibm.com Abstract This paper explores the use of a character segment based character correction model, language modeling, and shallow morphology for Arabic OCR error cor- rection. Experimentation shows that character segment based correction is su- perior to single character correction and that language modeling boosts correction, by improving the ranking of candidate corrections, while shallow morphology had a small adverse effect. Further, given sufficiently large corpus to extract a dictionary and to train a language model, word based correction works well for a morphologically rich language such as Arabic. 1 Introduction Recent advances in printed document digitization and processing led to large scale digitization ef- forts of legacy printed documents producing document images. To enable subsequent proc- essing and retrieval, the document images are often transformed to character-coded text using Optical Character Recognition (OCR). Although OCR is fast, OCR output typically contains er- rors. The errors are even more pronounced in OCR’ed Arabic text due to Arabic’s orthographic and morphological properties. The introduced errors adversely affect linguistic processing and retrieval of OCR’ed documents. This paper ex- plores the effectiveness post-OCR error correc- tion. The correction uses an improved character segment based noisy channel model, language modeling, and shallow morphological processing to correct OCR errors. The paper will be organ- ized as follows: Section 2 provides background information on Arabic OCR and OCR error cor- rection; Section 3 presents the error correction methodology; Section 4 reports and discusses experimental results; and Section 5 concludes the paper and provides possible future directions. 2 Background This section reviews prior work on Arabic OCR for Arabic and OCR error correction. 2.1 Arabic OCR The goal of OCR is to transform a document im- age into character-coded text. The usual process is to automatically segment a document image into character images in the proper reading order using image analysis heuristics, apply an auto- matic classifier to determine the character codes that most likely correspond to each character im- age, and then exploit sequential context (e.g., preceding and following characters and a list of possible words) to select the most likely charac- ter in each position. The character error rate can be influenced by reproduction quality (e.g., original documents are typically better than pho- tocopies), the resolution at which a document was scanned, and any mismatch between the in- stances on which the character image classifier was trained and the rendering of the characters in the printed document. Arabic OCR presents sev- eral challenges, including: • Arabic’s cursive script in which most charac- ters are connected and their shape vary with posi- tion in the word. • The optional use of word elongations and liga- tures, which are special forms of certain letter sequences. …

Content truncated to 500 words