Linguistic Error Correction of Japanese Sentences

LINGUISTIC ERROR CORRECTION OF JAPANESE SENTENCES Tsutomu Kawada, Shin-ya Amano and Kunio Sakai Information systems Lab., Research and Development Center, Toshiba Corporation 1 Komukai Toshiba-cho, Saiwai-ku, Kawasaki 210, JAPAN Abstract This paper describes a newly developed lin- guistic error correction system, which can cor- rect errors and rejections of Japanese sentences by using linguistic knowledge. Conventional optical character readers (OCR) need human assistance to correct their recognition errors and rejections. An operator must teach the OCR correct answers whenever an illegible character pattern occurs. If this error correction operation is mechanized, the throughput of the OCR will increase. This linguistic error correction system of- fers means of automated error correction by an- alysing sentences of the OCR outputs linguisti- cally. This system grammatically selects legal letters from the candidates which can not be decided uniquely by pattern recognition only, and recommends grammatically and semantically meaningful letters for the illegible letter. i. Introduction More than 2,000 different chinese charac- ters are used in Japanese newspapers and pub- lications. The large repertory of letters as well as the structual complexity of each char- acter pattern are the main difficulties to rec- ognize letters by OCR. Mutually similar char- acter patterns mainly cause recognition errors and rejections. The difference between such similar letters usually concentrate on a local area [ ~ .... ~ , ~ .... ~ , {~ .... , j~ .... j~ , :k .... ~,...:~ , eta]. The output of OCR contains these ambi- guities. It is necessary to use the contextual in- formation contained in every kind of natural text. The application of context makes it pos- sible to detect errors or even to correct them. The main object of this error correction system is to resolve these ambiguities on the basis of linguistic knowledge. To resolve these ambi- guities each ambiguous letter is put in its sentence, and the sentence is analyzed syntac- tically and semantically. When the sentences is acceptable the letter is selected preferably. There has been some contextual post pro- cessing systems. 1~2 But these systems are de- signed to read only postal-addresses. The words which are included in the postal-address are restricted within a relatively narrow do- main. The postal address usually consists of a person's name, place name, postal code and num- bers. On the other hand this newly developed kanji OCR can read actual texts of Japanese sentences, and its contextual error correction system of it has been designed to deal with many kinds of words of various parts of speech (noun, verb, adjective, adverb, pronoun, etc.). This error correction system has a practical 25,000 words dictionary, and can correct 53.8 percent of the errors which are included in the outputs of kanji OCR. 2. Restrictions This system imposes two restrictions on input data. One is that the input must consist of grammatical Japanese sentences in order that syntax analysis can be applicable. This system is not effective for only numeral data texts or a mere list of words. The other restriction …

Content truncated to 500 words