Data-Driven Response Generation in Social Media

Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 583–593, Edinburgh, Scotland, UK, July 27–31, 2011. c⃝2011 Association for Computational Linguistics Data-Driven Response Generation in Social Media Alan Ritter Computer Sci. & Eng. University of Washington Seattle, WA 98195 aritter@cs.washington.edu Colin Cherry National Research Council Canada Ottawa, Ontario, K1A 0R6 Colin.Cherry@nrc-cnrc.gc.ca William B. Dolan Microsoft Research Redmond, WA 98052 billdol@microsoft.com Abstract We present a data-driven approach to generat- ing responses to Twitter status posts, based on phrase-based Statistical Machine Translation. We find that mapping conversational stimuli onto responses is more difficult than translat- ing between languages, due to the wider range of possible responses, the larger fraction of unaligned words/phrases, and the presence of large phrase pairs whose alignment cannot be further decomposed. After addressing these challenges, we compare approaches based on SMT and Information Retrieval in a human evaluation. We show that SMT outperforms IR on this task, and its output is preferred over actual human responses in 15% of cases. As far as we are aware, this is the first work to investigate the use of phrase-based SMT to di- rectly translate a linguistic stimulus into an ap- propriate response. 1 Introduction Recently there has been an explosion in the number of people having informal, public conversations on social media websites such as Facebook and Twit- ter. This presents a unique opportunity to build collections of naturally occurring conversations that are orders of magnitude larger than those previously available. These corpora, in turn, present new op- portunities to apply data-driven techniques to con- versational tasks. We investigate the problem of response genera- tion: given a conversational stimulus, generate an appropriate response. Specifically, we employ a large corpus of status-response pairs found on Twit- ter to create a system that responds to Twitter status posts. Note that we make no mention of context, in- tent or dialogue state; our goal is to generate any re- sponse that fits the provided stimulus; however, we do so without employing rules or templates, with the hope of creating a system that is both flexible and extensible when operating in an open domain. Success in open domain response generation could be immediately useful to social media plat- forms, providing a list of suggested responses to a target status, or providing conversation-aware auto- complete for responses in progress. These features are especially important on hand-held devices (Has- selgren et al., 2003). Response generation should also be beneficial in building “chatterbots” (Weizen- baum, 1966) for entertainment purposes or compan- ionship (Wilks, 2006). However, we are most ex- cited by the future potential of data-driven response generation when used inside larger dialogue sys- tems, where direct consideration of the user’s utter- ance could be combined with dialogue state (Wong and Mooney, 2007; Langner et al., 2010) to generate locally coherent, purposeful dialogue. In this work, we investigate statistical machine translation as an approach for response generation. We are motivated by the following observation: In naturally occurring discourse, there is often a strong …

Content truncated to 500 words