An Automatic Chinese Document Revision System Using Bit and Character Mask Approach

Language, Information and Computation (PACLIC12), 18-20 Feb, 1998, 283-294 An Automatic Chinese Document Revision System Using Bit and Character Mask Approach June-Jei Kuo Matsushita Electric Institute of Technology (Taipei) Co., Ltd. Rm.1002, 10 FL, No.136, Sec. 3, Jen-Ai Road, Taipei, Taiwan, R.O.C. kaku@mitt.com.tw Abstract The errors in Chinese document are mainly caused in two stages - input and editing. There are homonyms or homophones selection error, ambiguous pronunciation error, word segmentation error, similar shape character error, editing operation error and so on. In order to increase the quality of Chinese text, the conventional Chinese document revision system used the similar characters set and language model with some statistical date. Nevertheless, there are the following problems: (1) The perfect similar character set is difficult to make (2) Due to the copyright problem the large and balanced Chinese corpus is very difficult to be obtained (3) The above editing errors can not be solved simultaneously (4) The average success revision rate is not over 75%. In this paper we study the Chinese features and phonetic-input-to-character conversion system for Chinese. It is found that the Chinese phonetic information and the related conversion algorithm are much help to detect and revise the input errors in Chinese document. As to the editing errors, a special code structure of Chinese pronunciation which has only one bit difference among similar pronunciations is proposed In addition, the bits and characters mask technology is also proposed respectively. The experimental result of the proposed system show that the average success revision rate of the proposed system is close to 87%. Keywords: Natural language processing, document revision, bit mask,character mask, phonetic information, dynamic programming 1. Introduction As the rapid progress of computer technology, the need for document computerization is also rised. So, how to enhance the document quality in the computer becomes an important issue. In addition, the document revision technology will be needed as a preprocessor or postprocessor in many application softwares, such as character recognition, speech recognition, machine translation and so on. Therefore, there are many approaches [1] on automatic document revision. The document revision can be further divided into error detection and. error modification. Nevertheless, the document error types is dependent on its processing language. For example, the English spelling checker can not be used to revise Chinese or Japanese document due to the different error types. Thus, the error types e of Chinese document will be described below. The error types of Chinese document can be divided into two parts. The first part is caused by input stage and the other part is caused by editing stage. Moreover, in input stage the different input way, such as phonetic way or radical way, will also have different error types. 283 Language, Information and Computation (PACLIC12), 18-20 Feb, 1998, 283-294 In the input part: (1) Using phonetic way 1.Ambiguous pronunciation error The phonetic symbols of a Chinese character can be divided into four parts: consonant, intermediate, vowel and tones. The possible ambiguous phonetic symbols in each part will be …

Content truncated to 500 words