The Utility of Discourse Structure in Identifying Resolved Threads in Technical User Forums

Proceedings of COLING 2012: Technical Papers, pages 2739–2756, COLING 2012, Mumbai, December 2012. The Utility of Discourse Structure in Identifying Resolved Threads in Technical User Forums Li Wang1,2 Su Nam Kim3 Timothy Baldwin1,2 (1) Dept. of Computing and Information Systems, The University of Melbourne (2) NICTA Victoria Research Laboratory (3) Faculty of Information Technology, Monash University li@liwang.info, sunamkim@gmail.com, tb@ldwin.net ABSTRACT Online discussion forums are a valuable means for users to resolve specific information needs, both interactively for the participants and statically for users who search/browse over historical thread data. However, the complex structure of forum threads can make it difficult for users to extract relevant information. Automatically identifying whether the problem in a thread has been solved or not can help direct users to threads where the original problem has been solved, hence enhancing their prospects of solving their particular problem. In this paper, we investigate the task of Solvedness classification by exploiting the discourse structure of forum threads. Experimental results show that simple features derived from thread discourse structure can greatly boost the accuracy of Solvedness classification, which has been shown to be very difficult in previous research. KEYWORDS: Discourse Structure, Web User Forum, Social Media, Dialogue Act. 2739 1 Introduction Web user forums (or simply “forums”) are online platforms for people to discuss information and obtain information via a text-based threaded discourse, generally in a pre-determined domain (e.g. IT support or DSLR cameras). With the advent of Web 2.0, there has been an explosion of web authorship in this area, and forums are now widely used in various areas such as customer support, community development, interactive reporting and online eduction. In addition to providing the means to interactively participate in discussions or obtain/provide answers to questions, the vast volumes of data contained in forums make them a valuable resource for “support sharing”, i.e. looking over records of past user interactions to potentially find an immediately applicable solution to a current problem. On the one hand, more and more answers to questions over a wide range of domains are becoming available on forums; on the other hand, it is becoming harder and harder to extract and access relevant information due to the sheer scale and diversity of the data. In the domain of troubleshooting-oriented forums, one potential way to enhance information access and support sharing is to automatically identify threads where the original information need has been resolved. By filtering out threads which do not contain a valid answer, we can focus the attention of users on threads which have a greater chance of containing the required solution. Baldwin et al. (2007) explore this task of Solvedness classification, and find that it is an extremely difficult problem. Figure 1 shows an example thread, made up of 5 posts from 3 distinct participants, from the ILIAD (Improved Linux Information Access by Data Mining) dataset of Baldwin et al. (2007). In this thread, Post1 and Post3 are both from the thread’s initiator UserA. Post1 asks a question, and Post3 asks for …

Content truncated to 500 words