Finding Problem Solving Threads in Online Forum

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 1413–1417, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Finding Problem Solving Threads in Online Forum Zhonghua Qu and Yang Liu The University of Texas at Dallas {qzh,yangl}@hlt.utdallas.edu Abstract Online forum is an important source that people use to find answers to questions. Most search engines simply retrieve “rele- vant” threads, but those are not necessarily good threads in terms of providing qual- ity answers. In this paper we propose a two-step approach to classify online forum threads according to their informativeness in terms of question answering. We use statistical models to first categorize posts inside a thread. Then, a variety of features including post level information and other meta-data information are used to classify the thread. We show promising results us- ing the online support forum data we have collected. 1 Introduction Online forums often contain useful information, such as answers to questions. When people use search engines to search the forum for answers, the returned threads can be of very low quality in terms of providing informative answers. Some threads are either long conversations without any final conclusive answers or contain answers that do not work. This is especially the case in tech- nical forums. There has been some work on ex- tracting information from online forums. For ex- ample, Cong et al. (Cong et al., 2008) used la- beled sequential patterns to detect question sen- tences in online forums. Ding et al. (Ding et al., 2008) used Conditional Random Fields to extract context of questions for answer detection. Huang et al. (Huang et al., 2007) used SVM to automati- cally extract and rank title-reply pairs from online discussion forums for chatbot knowledge. These previous studies provide a relatively good founda- tion for answer finding and extraction. In online environments like mailing list or forums, question answering is carried out in conversations. In this paper we take advantage of the structure of con- versation and language cues to answer a basic yet important question: Does the thread actually solve the problem? We will make use of the conversation within the thread to determine how users think of the an- swers. To classify the usefulness of a thread, we propose a two-step approach using statistical mod- els. First we classify the posts inside a thread into different categories (e.g., problem description, so- lution, feedback). Different models using both content and contextual information are evaluated. Then, we develop features generated from the post categories together with forum meta-data to clas- sify a thread’s usefulness. In our collection of fo- rum threads, we show that our proposed methods achieve good results, significantly better than the rule-based baseline. 2 Data Collection and Annotation We created our own data collection and annota- tion.1 We crawled 20,000 threads from an ac- tive online forum (Oracle database support forum – general section). As an initial study, we se- lected 200 threads randomly and asked two anno- tators to label the threads …

Content truncated to 500 words