The Postprocessing of Optical Character Recognition Based on Statistical Noisy Channel and Language Model

The Postprocessing of Optical Character Recognition based on Statistical Noisy channel and language model Jason J. S. Chang and Shun-Der Chen Department of computer Science , National Tsing Hua University e-mail jschang@cs.nthu.edu.tw dr798306®cs.nthu.edu.tw Abstract The techniques of image processing have been used in optical character recognition (OCR) for a long time. The recognition method evolved from early "pattern recognition" to "feature extraction" recently. The recognition rate is raised from 70% to 90%. But the character by character recognition technique has its limitation. Using language models to assist the OCR system in improving recognition rate is the topic of many recent researches. Recently, the related research on Chinese nature language processing has improved rapidly. These improvement include the Chinese word segmentation, syntax analysis, semantic analysis, collocation analysis, statistical language models. In this paper, we will propose a new techniques for Chinese OCR postprocessing and postediting. We combine noisy channel model and the technique of natural language processing to implement an OCR postprocessing system. From the result of experiments, we found noisy channel model very effective for postprocessing. Under the approach, it is possible to recover the correct character, even when it is not in the candidate list produced by the OCR system. 1. Introduction Recently, the research of human-machine interface is becoming more and more important. To reduce the workload of user entering text and data into a computer system, there are many human-machine interfaces developed, such as optical character recognition (OCR), speech recognition, etc. However, some errors are usually introduced in the recognition process. Therefore, means for finding and correcting such errors in indispensable. Research in character recognition for Chinese faces more difficulties than for other languages. Firstly, Chinese has a huge character set. Secondly, Chinese characters have more complex structure than alphabetic characters and there are a large number of similar character groups. Therefore, we need some kind of contextual information to detect and correct errors. Language models can be used to provide such contextual information to enhance the correction rate and speed of a man-machine interface system. For example, Tsuyoshi [14] use character frequency and the morphological analysis to improve the handwritten Japanese OCR system. 2000 - 1500 1000 c.) 500 - 0 0 0, en� VI OS ei N cn co) cot error count /100 'error correct e 00 1/1 Recently the techniques of Chinese OCR have advanced greatly, the recognition rate has reached 95% and 90% for printed and handwritten text respectively. In view of the limitation of the techniques of OCR, we hope to use natural language models to detect the 5 to 10% errors and suggested their possible correction in a postediting environment. With an effective postprocessing technique and user-friendly posteditor, the usability of OCR technology can be greatly enhanced. 2. The framework of the System Image Recognition Unit Post Processing Unit Chinese Print Text data>/ Chinese OCR Recognition System ��Image Statistical model Detection Natural Language Text Filc.< Correction Model Noisy Channel Model In this paper, we have adopted a multi-stage postprocessing approach. In …

Content truncated to 500 words