Analysing Human Strategies of Information Transmission as a Function of Discourse Context

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 647–660 November 10–11, 2021. ©2021 Association for Computational Linguistics 647 Analysing Human Strategies of Information Transmission as a Function of Discourse Context Mario Giulianelli and Raquel Fernández Institute for Logic, Language and Computation University of Amsterdam {m.giulianelli|raquel.fernandez}@uva.nl Abstract Speakers are thought to use rational infor- mation transmission strategies for efficient communication (Genzel and Charniak, 2002; Aylett and Turk, 2004; Jaeger and Levy, 2007). Previous work analysing these strategies in sentence production has failed to take into account how the information content of sen- tences varies as a function of the available discourse context. In this study, we estimate sentence information content within discourse context. We find that speakers transmit in- formation at a stable rate—i.e., rationally—in English newspaper articles but that this rate decreases in spoken open domain and writ- ten task-oriented dialogues. We also observe that speakers’ choices are not oriented towards local uniformity of information, which is an- other hypothesised rational strategy. We sug- gest that a more faithful model of communica- tion should explicitly include production costs and goal-oriented rewards. 1 Introduction Linguistic communication can be understood as information exchange through a noisy channel. Speakers are sensitive to the properties of the chan- nel in two ways (Clark and Wilkes-Gibbs, 1986; Clark and Schaefer, 1989). On the one hand, they try to reduce the processing effort of the addressee. For example, in the absence of established dis- course context, speakers can produce utterances that are easier to process in order to minimise the chance of transmission error. On the other hand, speakers try to reduce their own production effort. For example, given a fixed amount of information that they intend to transmit, speakers can take the risk of producing more concise utterances that are less costly from the production point of view, and expect the addressee to exploit the utterance con- text for interpretation. Effective and efficient infor- mation exchange under these two competing pres- sures can be modelled using the tools of Informa- tion Theory (Shannon, 1948). Indeed, information- theoretic models have successfully accounted for surprisal in speech perception (Jelinek et al., 1975; Clayards et al., 2008), reading (Keller, 2004; Dem- berg and Keller, 2008; Levy et al., 2009), sentence interpretation (Levy, 2008; Gibson et al., 2013), and overlap in turn taking (Dethlefs et al., 2016). The information content of a sentence H(S)— i.e., its entropy or the effort it takes to process it out of context—and the informativeness of its discourse context I(S; C) are hypothesised to be related. According to the principle of Entropy Rate Constancy (ERC; Genzel and Charniak, 2002), as discourse develops, these two quantities increase at a similar rate; thus, the difference between them— i.e., the effort that it takes to process a sentence in context—remains constant over the course of a discourse: H(S|C) ≡ H(S) − I(S; C). A slight relaxation of this prediction is that the informa- tion content of a sentence in context remains uni- form, …

Content truncated to 500 words