Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 324–333, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing Brian Roark† Asaf Bachrach‡ Carlos Cardenas◦ and Christophe Pallier‡ †Center for Spoken Language Understanding, Oregon Health & Science University ‡ INSERM-CEA Cognitive Neuroimaging Unit, Gif sur Yvette, France ◦MIT roark@cslu.ogi.edu asafbac@gmail.com cardenas@mit.edu christophe@pallier.org Abstract A number of recent publications have made use of the incremental output of stochastic parsers to derive measures of high utility for psycholinguistic modeling, following the work of Hale (2001; 2003; 2006). In this paper, we present novel methods for calculating separate lexical and syntactic surprisal measures from a single incremental parser using a lexical- ized PCFG. We also present an approx- imation to entropy measures that would otherwise be intractable to calculate for a grammar of that size. Empirical results demonstrate the utility of our methods in predicting human reading times. 1 Introduction Assessment of linguistic complexity has played an important role in psycholinguistics and neu- rolinguistics for a long time, from the use of mean length of utterance and related scores in child language development (Klee and Fitzgerald, 1985), to complexity scores related to reading dif- ficulty in human sentence processing studies (Yn- gve, 1960; Frazier, 1985; Gibson, 1998). Opera- tionally, such linguistic complexity scores are de- rived via deterministic manual (human) annotation and scoring algorithms of language samples. Nat- ural language processing has been employed to automate the extraction of such measures (Sagae et al., 2005; Roark et al., 2007), which can have high utility in terms of reduction of time required to annotate and score samples. More interest- ingly, however, novel data driven methods are be- ing increasingly employed in this sphere, yield- ing language sample characterizations that require NLP in their derivation. For example, scores derived from variously estimated language mod- els have been used to evaluate and classify lan- guage samples associated with neurodevelopmen- tal or neurodegenerative disorders (Roark et al., 2007; Solorio and Liu, 2008; Gabani et al., 2009), as well as within general studies of human sen- tence processing (Hale, 2001; 2003; 2006). These scores cannot feasibly be derived by hand, but rather rely on large-scale statistical models and structured inference algorithms to be derived. This is quickly becoming an important application of NLP, making possible new methods in the study of human language processing in both typical and impaired populations. The use of broad-coverage parsing for psy- cholinguistic modeling has become very popular recently. Hale (2001) suggested a measure (sur- prisal) derived from an Earley (1970) parser us- ing a probabilistic context-free grammar (PCFG) for psycholinguistic modeling; and in later work (Hale, 2003; 2006) he suggested an alternate parser-derived measure (entropy reduction) that may also account for some human sentence pro- cessing performance. Recent work continues to advocate surprisal in particular as a very use- ful measure for predicting processing difficulty (Boston et al., 2008a; Boston et al., 2008b; Dem- berg and …

Content truncated to 500 words