Unsupervised Modeling of Twitter Conversations

Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the ACL, pages 172–180, Los Angeles, California, June 2010. c⃝2010 Association for Computational Linguistics Unsupervised Modeling of Twitter Conversations Alan Ritter∗ Computer Sci. & Eng. University of Washington Seattle, WA 98195 aritter@cs.washington.edu Colin Cherry∗ National Research Council Canada Ottawa, Ontario, K1A 0R6 Colin.Cherry@nrc-cnrc.gc.ca Bill Dolan Microsoft Research Redmond, WA 98052 billdol@microsoft.com Abstract We propose the first unsupervised approach to the problem of modeling dialogue acts in an open domain. Trained on a corpus of noisy Twitter conversations, our method dis- covers dialogue acts by clustering raw utter- ances. Because it accounts for the sequential behaviour of these acts, the learned model can provide insight into the shape of communica- tion in a new medium. We address the chal- lenge of evaluating the emergent model with a qualitative visualization and an intrinsic con- versation ordering task. This work is inspired by a corpus of 1.3 million Twitter conversa- tions, which will be made publicly available. This huge amount of data, available only be- cause Twitter blurs the line between chatting and publishing, highlights the need to be able to adapt quickly to a new medium. 1 Introduction Automatic detection of dialogue structure is an im- portant first step toward deep understanding of hu- man conversations. Dialogue acts1 provide an initial level of structure by annotating utterances with shallow discourse roles such as “statement”, “question” and “answer”. These acts are useful in many applications, including conversational agents (Wilks, 2006), dialogue systems (Allen et al., 2007), dialogue summarization (Murray et al., 2006), and flirtation detection (Ranganath et al., 2009). Dialogue act tagging has traditionally followed an annotate-train-test paradigm, which begins with the ∗This work was conducted at Microsoft Research. 1Also called “speech acts” design of annotation guidelines, followed by the col- lection and labeling of corpora (Jurafsky et al., 1997; Dhillon et al., 2004). Only then can one train a tag- ger to automatically recognize dialogue acts (Stol- cke et al., 2000). This paradigm has been quite suc- cessful, but the labeling process is both slow and expensive, limiting the amount of data available for training. The expense is compounded as we con- sider new methods of communication, which may require not only new annotations, but new annota- tion guidelines and new dialogue acts. This issue be- comes more pressing as the Internet continues to ex- pand the number of ways in which we communicate, bringing us e-mail, newsgroups, IRC, forums, blogs, Facebook, Twitter, and whatever is on the horizon. Previous work has taken a variety of approaches to dialogue act tagging in new media. Cohen et al. (2004) develop an inventory of dialogue acts specific to e-mail in an office domain. They design their in- ventory by inspecting a large corpus of e-mail, and refine it during the manual tagging process. Jeong et al. (2009) use semi-supervised learning to transfer dialogue acts from labeled speech corpora to the In- ternet media of forums and e-mail. They manually restructure the …

Content truncated to 500 words