A Dataset for Research on Short-Text Conversations

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 935–945, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Dataset for Research on Short-Text Conversation ∗ Hao Wang§ Zhengdong Lu‡ Hang Li‡ Enhong Chen§ § xdwangh@mail.ustc.edu.cn ‡lu.zhengdong@huawei.com ‡hangli.hl@huawei.com §cheneh@ustc.edu.cn §Univ. of Sci & Tech of China, China ‡Noah’s Ark Lab, Huawei Technologies, Hong Kong Abstract Natural language conversation is widely re- garded as a highly difficult problem, which is usually attacked with either rule-based or learning-based models. In this paper we propose a retrieval-based automatic response model for short-text conversation, to exploit the vast amount of short conversation in- stances available on social media. For this purpose we introduce a dataset of short-text conversation based on the real-world instances from Sina Weibo (a popular Chinese mi- croblog service), which will be soon released to public. This dataset provides rich collec- tion of instances for the research on finding natural and relevant short responses to a given short text, and useful for both training and test- ing of conversation models. This dataset con- sists of both naturally formed conversation- s, manually labeled data, and a large repos- itory of candidate responses. Our prelimi- nary experiments demonstrate that the simple retrieval-based conversation model performs reasonably well when combined with the rich instances in our dataset. 1 Introduction Natural language conversation is one of the holy grail of artificial intelligence, and has been taken as the original form of the celebrated Turing test. Pre- vious effort in this direction has largely focused on analyzing the text and modeling the state of the con- versation through dialogue models, while in this pa- ∗The work is done when the first author worked as intern at Noah’s Ark Lab, Huawei Techologies. per we take one step back and focus on a much easi- er task of finding the response for a given short text. This task is in clear contrast with previous effort in dialogue modeling in the following two aspects • we do not consider the context or history of conversations, and assume that the given short text is self-contained; • we only require the response to be natural, rel- evant, and human-like, and do not require it to contain particular opinion, content, or to be of particular style. This task is much simpler than modeling a complete dialogue session (e.g., as proposed in Turing test), and probably not enough for real conversation sce- nario which requires often several rounds of interac- tions (e.g., automatic question answering system as in (Litman et al., 2000)). However it can shed impor- tant light on understanding the complicated mecha- nism of the interaction between an utterance and it- s response. The research in this direction will not only instantly help the applications of short session dialogue such as automatic message replying on mo- bile phone and the chatbot employed in voice assis- tant like Siri1, but also it will eventually benefit the modeling of dialogues in a more general setting. Previous …

Content truncated to 500 words