A Probabilistic Earley Parser as a Psycholinguistic Model

A Probabilistic Earley Parser as a Psycholinguistic Model John Hale Department of Cognitive Science The Johns Hopkins University 3400 North Charles Street; Baltimore MD 21218-2685 hale@cogsci.jhu.edu Abstract In human sentence processing, cognitive load can be defined many ways. This report considers a defini- tion of cognitive load in terms of the total probability of structural options that have been disconfirmed at some point in a sentence: the surprisal of word wi given its prefix w0...i−1 on a phrase-structural lan- guage model. These loads can be efficiently calcu- lated using a probabilistic Earley parser (Stolcke, 1995) which is interpreted as generating predictions about reading time on a word-by-word basis. Un- der grammatical assumptions supported by corpus- frequency data, the operation of Stolcke’s probabilis- tic Earley parser correctly predicts processing phe- nomena associated with garden path structural am- biguity and with the subject/object relative asym- metry. Introduction What is the relation between a person’s knowledge of grammar and that same person’s application of that knowledge in perceiving syntactic structure? The answer to be proposed here observes three principles. Principle 1 The relation between the parser and grammar is one of strong competence. Strong competence holds that the human sentence processing mechanism directly uses rules of gram- mar in its operation, and that a bare minimum of extragrammatical machinery is necessary. This hy- pothesis, originally proposed by Chomsky (Chom- sky, 1965, page 9) has been pursued by many re- searchers (Bresnan, 1982) (Stabler, 1991) (Steed- man, 1992) (Shieber and Johnson, 1993), and stands in contrast with an approach directed towards the discovery of autonomous principles unique to the processing mechanism. Principle 2 Frequency affects performance. The explanatory success of neural network and constraint-based lexicalist theories (McClelland and St. John, 1989) (MacDonald et al., 1994) (Tabor et al., 1997) suggests a statistical theory of language performance. The present work adopts a numerical view of competition in grammar that is grounded in probability. Principle 3 Sentence processing is eager. “Eager” in this sense means the experimental situa- tions to be modeled are ones like self-paced reading in which sentence comprehenders are unrushed and no information is ignored at a point at which it could be used. The proposal is that a person’s difficulty per- ceiving syntactic structure be modeled by word-to- word surprisal (Attneave, 1959, page 6) which can be directly computed from a probabilistic phrase- structure grammar. The approach taken here uses a parsing algorithm developed by Stolcke. In the course of explaining the algorithm at a very high level I will indicate how the algorithm, interpreted as a psycholinguistic model, observes each principle. After that will come some simulation results, and then a conclusion. 1 Language models Stolcke’s parsing algorithm was initially applied as a component of an automatic speech recognition sys- tem. In speech recognition, one is often interested in the probability that some word will follow, given that a sequence of words has been seen. Given some lexicon of all possible words, a language model as- signs a probability to every …

Content truncated to 500 words