A New Predictive Analyzer of English

A New Predictive Analyzer of English Hiroyuki Musha Department of Information Science Tokyo Institute of Technology Ohokayama, Meguro-ku, Tokyo 152, JAPAN ABSTRACT Aspects of syntactic predictions made during the recognition of English sentences are investigated. We reinforce Kuno's origi- nal predictive analyzer[i] by introducing five types of predictions. For each type of prediction, we discuss and present its necessity, its description method, and recognition mechanism. We make use of three kinds of stacks whose behavior is specified by grammar rules in an extended version of Greibach normal form. We also investi- gate other factors that affect the predictive recognition process, i.e., preferences among syntactic ambiguities and necessary amount of lookahead. These factors as well as the proposed handling mechanisms of predictions are tested by analyzing two kinds of articles. In our experiment, more than seventy percent of sen- tences are recognized and looking two words ahead seems to be the critical length for the predictive recognition. 1. Introduction When human reads normal sentences, we rarely feel some- thing is wrong with the structure we are constructing and are sel- dom compelled to backtrack for reconstructing an alternative. If we could simulate the internal mechanism that makes it possible to select deterministically the unique syntactic structure in a simple way, we may be able to construct more natural and efficient language processing systems. In this paper, we focus our attention on syntax of natural languages, particularly English, and predic- tions or expectations that can be made solely with syntactic infor- mation during the sentence recognition process are analyzed in detail. It includes machine executable mechanisms that enable proper handling of analyzed aspects and a description method of the mechanisms as grammar rules. The recognition method can be seen as a deterministic one [2] if we permit looking some words ahead. Also included in this paper are results of an experimental analysis in which more than seventy percent of sentences are recog- nized. An analyzer which gives special attention to predictions was once developed by Kuno [1]. The analyzer makes use of the sim- ple stack mechanism whose behavior is specified by rules described in Greibach normal form. In the method, however, we can find several kinds of rules that do not correspond to human predictive recognition process, which will be pointed out in this paper. The following discussion is based mainly on the author's (sub- jective) retrospect of the recognition process of English sentences. The author's mother tongue is Japanese and he has been learning English as a second language. It seems to the author that he can understand better how he recognizes English than how he recog- nizes Japanese since he has been learning English consciously and can observe rather objectively the process of recognition. The rest of this paper is organized as follows. In the next section, we discuss aspects of predictions, laying stress upon their proper handling by computers. The following section presents the results of an experiment. The conclusions are presented in the last section. 2. Aspects …

Content truncated to 500 words