Entropy Converges Between Dialogue Participants: Explanations from an Information-Theoretic Perspective

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 537–546, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Entropy Converges Between Dialogue Participants: Explanations from an Information-Theoretic Perspective Yang Xu and David Reitter College of Information Sciences and Technology The Pennsylvania State University University Park, PA 16802, USA yang.xu@psu.edu, reitter@psu.edu Abstract The applicability of entropy rate constancy to dialogue is examined on two spoken di- alogue corpora. The principle is found to hold; however, new entropy change pat- terns within the topic episodes of dialogue are described, which are different from written text. Speaker’s dynamic roles as topic initiators and topic responders are associated with decreasing and increasing entropy, respectively, which results in lo- cal convergence between these speakers in each topic episode. This implies that the sentence entropy in dialogue is con- ditioned on different contexts determined by the speaker’s roles. Explanations from the perspectives of grounding theory and interactive alignment are discussed, re- sulting in a novel, unified information- theoretic approach of dialogue. 1 Introduction Information in written text and speech is strate- gically distributed. It has been claimed to be ordered such that the rate of information is not only close to the channel capacity, but also ap- proximately constant (Genzel and Charniak, 2002, 2003; Jaeger, 2010); these results were devel- oped within the framework of Information Theory (Shannon, 1948). In these studies, the per-word cross-entropy of a sentence is used to model the amount of information transmitted. Language is treated as a series of random variables of words. Most existing work examined written text as opposed to speech. Spoken dialogue is different from written text in many ways. For example, di- alogue contains more irregular or ungrammatical components, such as incomplete utterances, dis- fluencies etc. (Jurafsky and Martin, 2014, ch 12), which are “theoretically uninterested complexities that are unwanted” (Pickering and Garrod, 2004). Dialogue is also different from written text in high level discourse structure. The paragraphs in writ- ten text, which function as relatively standalone topic units, are constructed under the guidance of one consistent author. On the other hand, the con- stitution and transformation of topics in dialogue are more dynamic processes, which are the result of the joint activity from multiple speakers (Linell, 1998). In nature, written text is a monologue, while dialogue is a joint activity (Clark, 1996). From the application perspective, investigating entropy in dialogue can help us better understand which speaker contributes the most information, and thus may potentially benefit tasks such as con- versational roles identification (Traum, 2003) etc. From the theoretical perspective, we believe that such investigation will reveal some unique fea- tures of the formation of higher level discourse structure in dialogue that are different from writ- ten text, e.g., topic episode shifts, because pre- vious studies have found the correlation between entropy decrease and potential topic shift in writ- ten text (Qian and Jaeger, 2011). Finally, entropy is closely related to predictability and processing demands, which has implications for cognitive …

Content truncated to 500 words