Shared common ground influences information density in microblog texts

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 1587–1596, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Shared common ground influences information density in microblog texts Gabriel Doyle Dept. of Psychology Stanford University Stanford, CA, USA, 94305 gdoyle@stanford.edu Michael C. Frank Dept. of Psychology Stanford University Stanford, CA, USA, 94305 mcfrank@stanford.edu Abstract If speakers use language rationally, they should structure their messages to achieve approximately uniform information density (UID), in order to optimize transmission via a noisy channel. Previous work identified a consistent increase in linguistic information across sentences in text as a signature of the UID hypothesis. This increase was derived from a predicted increase in context, but the context itself was not quantified. We use microblog texts from Twitter, tied to a sin- gle shared event (the baseball World Series), to quantify both linguistic and non-linguistic context. By tracking changes in contextual information, we predict and identify grad- ual and rapid changes in information content in response to in-game events. These find- ings lend further support to the UID hypoth- esis and highlights the importance of non- linguistic common ground for language pro- duction and processing. 1 Introduction There are many ways express a given message in natural language, so how do speakers decide be- tween potential structures? One prominent hypothe- sis is that they aim for structures that best convey the intendeed message in the context of the communi- cation. On this view, the use of natural languages is assumed to follow optimal information transmission results from information theory (Shannon, 1948). In particular, speakers should structure their mes- sages to approximate uniform information density across symbols (words and phonemes), which is optimal for transmission of information through a noisy channel. At least three lines of evidence suggest that speak- ers do make choices to increase the uniformity of information density across their utterances. First, speakers phonologically reduce more predictable material (Aylett and Turk, 2004; Aylett and Turk, 2006; Bell et al., 2003). Second, they omit or reduce optional lexical material in cases where the subse- quent syntactic information is relatively more pre- dictable (Levy and Jaeger, 2007; Frank and Jaeger, 2008; Jaeger, 2010). Third, and most relevant to our current hypothesis, speakers appear to increase the complexity of their utterances as a discourse devel- ops (Genzel and Charniak, 2002; Genzel and Char- niak, 2003; Qian and Jaeger, 2012). We expand on this finding below. Following the UID hypothesis, Genzel and Char- niak 2002 proposed that H(Yi), the total entropy of part i of a message (e.g., a word) is constant. They compute this expression by considering Xi, the ran- dom variable representing the precise word that will appear at position i, conditioned on all the previ- ously observed words. They then further factor this expression into two terms: H(Yi) = H(Xi|Ci, Li) = H(Xi|Li) − I(Xi; Ci|Li) (1) where the first term H(Xi|Li) is the dependence of the current word on only the local …

Content truncated to 500 words