An Analysis of Frequency- and Memory-Based Processing Costs

Proceedings of NAACL-HLT 2013, pages 95–105, Atlanta, Georgia, 9–14 June 2013. c⃝2013 Association for Computational Linguistics An Analysis of Frequency- and Memory-Based Processing Costs Marten van Schijndel The Ohio State University vanschm@ling.osu.edu William Schuler The Ohio State University schuler@ling.osu.edu Abstract The frequency of words and syntactic con- structions has been observed to have a sub- stantial effect on language processing. This begs the question of what causes certain con- structions to be more or less frequent. A the- ory of grounding (Phillips, 2010) would sug- gest that cognitive limitations might cause lan- guages to develop frequent constructions in such a way as to avoid processing costs. This paper studies how current theories of working memory fit into theories of language process- ing and what influence memory limitations may have over reading times. Measures of such limitations are evaluated on eye-tracking data and the results are compared with predic- tions made by different theories of processing. 1 Introduction Frequency effects in language have been isolated and observed in many studies (Trueswell, 1996; Jurafsky, 1996; Hale, 2001; Demberg and Keller, 2008). These effects are important because they il- luminate the ontogeny of language (how individual speakers have acquired language), but they do not answer questions about the phylogeny of language (how the language came to its current form). Phillips (2010) has hypothesized that grammar rule probabilities may be grounded in memory lim- itations. Increased delays in processing center- embedded sentences as the number of embeddings increases, for example, are often explained in terms of a complexity cost associated with maintaining in- complete dependencies in working memory (Gib- son, 2000; Lewis and Vasishth, 2005). Other stud- ies have shown a link between processing delays and the low frequency of center-embedded construc- tions like object relatives (Hale, 2001), but they have not explored the source of this low frequency. A grounding hypothesis would claim that the low probability of generating such a structure may arise from an associated memory load. In this account, while these complexity costs may involve language- specific concepts such as referent or argument link- ing, the underlying explanation would be one of memory limitations (Gibson, 2000) or neural acti- vation (Lewis and Vasishth, 2005). This paper seeks to explore the different predic- tions made by these theories on a broad-coverage corpus of eye-tracking data (Kennedy et al., 2003). In addition, the current experiment seeks to isolate memory effects from frequency effects in the same task. The results show that memory load measures are a significant factor even when frequency mea- sures are residualized out. The remainder of this paper is organized as fol- lows: Sections 2 and 3 describe several frequency and memory measures. Section 4 describes a proba- bilistic hierarchic sequence model that allows all of these measures to be directly computed. Section 5 describes how these measures were used to predict reading time durations on the Dundee eye-tracking corpus. Sections 6 and 7 present results and discuss. 2 Frequency Measures 2.1 Surprisal One of the strongest predictors of …

Content truncated to 500 words