A Noisy-Channel Model of Human Sentence Comprehension under Uncertain Input

Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 234–243, Honolulu, October 2008. c⃝2008 Association for Computational Linguistics A noisy-channel model of rational human sentence comprehension under uncertain input Roger Levy Department of Linguistics University of California – San Diego 9500 Gilman Drive #0108 La Jolla, CA 92093-0108 rlevy@ling.ucsd.edu ∗ Abstract Language comprehension, as with all other cases of the extraction of meaningful struc- ture from perceptual input, takes places un- der noisy conditions. If human language comprehension is a rational process in the sense of making use of all available infor- mation sources, then we might expect uncer- tainty at the level of word-level input to af- fect sentence-level comprehension. However, nearly all contemporary models of sentence comprehension assume clean input—that is, that the input to the sentence-level com- prehension mechanism is a perfectly-formed, completely certain sequence of input tokens (words). This article presents a simple model of rational human sentence comprehension under noisy input, and uses the model to in- vestigate some outstanding problems in the psycholinguistic literature for theories of ra- tional human sentence comprehension. We argue that by explicitly accounting for input- level noise in sentence processing, our model provides solutions for these outstanding prob- lems and broadens the scope of theories of hu- man sentence comprehension as rational prob- abilistic inference. ∗Part of this work has benefited from presentation at the 21st annual meeting of the CUNY Sentence Processing Confer- ence in Chapel Hill, NC, 14 March 2008, and at a seminar at the Center for Research on Language, UC San Diego. I am grateful to Klinton Bicknell, Andy Kehler, and three anonymous review- ers for comments and suggestions, Cyril Allauzen for guidance regarding the OpenFST library, and to Mark Johnson, Mark- Jan Nederhof, and Noah Smith for discussion of renormalizing weighted CFGs. 1 Introduction Considering the adversity of the conditions under which linguistic communication takes place in ev- eryday life—ambiguity of the signal, environmental competition for our attention, speaker error, and so forth—it is perhaps remarkable that we are as suc- cessful at it as we are. Perhaps the leading expla- nation of this success is that (a) the linguistic sig- nal is redundant, and (b) diverse information sources are generally available that can help us obtain infer the intended message (or something close enough) when comprehending an utterance (Tanenhaus et al., 1995; Altmann and Kamide, 1999; Genzel and Char- niak, 2002, 2003; Aylett and Turk, 2004; Keller, 2004; Levy and Jaeger, 2007). Given the difficulty of this task coupled with the availability of redun- dancy and useful information sources, it would seem rational for all available information to be used to its fullest in sentence comprehension. This idea is either implicit or explicit in several interactivist the- ories of probabilistic language comprehension (Ju- rafsky, 1996; Hale, 2001; Narayanan and Jurafsky, 2002; Levy, 2008). However, these theories have implicitly assumed a partitioning of interactivity that distinguishes the word as a fundamental level of linguistic information processing: word recognition is an …

Content truncated to 500 words