Toward a Psycholinguistically-Motivated Model of Language Processing

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 785–792 Manchester, August 2008 Toward a Psycholinguistically-Motivated Model of Language Processing William Schuler Computer Science and Engineering University of Minnesota schuler@cs.umn.edu Samir AbdelRahman Department of Computer Science Cairo University s.abdelrahman@fci-cu.edu.eg Tim Miller Computer Science and Engineering University of Minnesota tmill@cs.umn.edu Lane Schwartz Computer Science and Engineering University of Minnesota lschwar@cs.umn.edu Abstract Psycholinguistic studies suggest a model of human language processing that 1) per- forms incremental interpretation of spo- ken utterances or written text, 2) preserves ambiguity by maintaining competing anal- yses in parallel, and 3) operates within a severely constrained short-term memory store — possibly constrained to as few as four distinct elements. This paper de- scribes a relatively simple model of lan- guage as a factored statistical time-series process that meets all three of the above desiderata; and presents corpus evidence that this model is sufficient to parse natu- rally occurring sentences using human-like bounds on memory. 1 Introduction Psycholinguistic studies suggest a model of human language processing with three important proper- ties. First, eye-tracking studies (Tanenhaus et al., 1995; Brown-Schmidt et al., 2002) suggest that hu- mans analyze sentences incrementally, assembling and interpreting referential expressions even while they are still being pronounced. Second, humans appear to maintain competing analyses in paral- lel, with eye gaze showing significant attention to competitors (referents of words with similar pre- fixes to the correct word), even relatively long af- ter the end of the word has been encountered, when attention to other distractor referents has fallen off (Dahan and Gaskell, 2007). Preserving ambigu- ity in a parallel, non-deterministic search like this may account for human robustness to missing, un- known, mispronounced, or misspelled words. Fi- nally, studies of short-term memory capacity sug- c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. gest human language processing operates within a severely constrained short-term memory store — possibly restricted to as few as four distinct ele- ments (Miller, 1956; Cowan, 2001). The first two observations may be taken to endorse existing probabilistic beam-search mod- els which maintain multiple competing analyses, pruned by contextual preferences and dead ends (e.g. Roark, 2001). But the last observation on memory bounds imposes a restriction that until now has not been evaluated in a corpus study. Can a simple, useful human-like processing model be defined using these constraints? This paper de- scribes a relatively simple model of language as a factored statistical time-series process that meets all three of the above desiderata; and presents corpus evidence that this model is sufficient to parse naturally occurring sentences using human- like bounds on memory. The remainder of this paper is organized as fol- lows: Section 2 describes some current approaches to incremental parsing; Section 3 describes a statis- tical framework for parsing using a bounded stack of explicit constituents; Section 4 describes an ex- periment to estimate the level of coverage of the Penn Treebank corpus that can be achieved with various …

Content truncated to 500 words