Global Thread-level Inference for Comment Classification in Community Question Answering

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 573–578, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Global Thread-Level Inference for Comment Classification in Community Question Answering Shafiq Joty, Alberto Barr´on-Cede˜no, Giovanni Da San Martino, Simone Filice, Llu´ıs M`arquez, Alessandro Moschitti, and Preslav Nakov, Qatar Computing Research Institute, HBKU {sjoty,albarron,gmartino,sfilice, lmarquez,amoschitti,pnakov}@qf.org.qa Abstract Community question answering, a recent evolution of question answering in the Web context, allows a user to quickly con- sult the opinion of a number of people on a particular topic, thus taking advantage of the wisdom of the crowd. Here we try to help the user by deciding automat- ically which answers are good and which are bad for a given question. In particular, we focus on exploiting the output struc- ture at the thread level in order to make more consistent global decisions. More specifically, we exploit the relations be- tween pairs of comments at any distance in the thread, which we incorporate in a graph-cut and in an ILP frameworks. We evaluated our approach on the benchmark dataset of SemEval-2015 Task 3. Results improved over the state of the art, confirm- ing the importance of using thread level in- formation. 1 Introduction Community question answering (CQA) is a recent evolution of question answering, in the Web con- text, where users pose questions and then receive answers from other users. This setup is very at- tractive, as the anonymity on the Web allows users to ask just about anything and then hope to get some honest answers from a number of people. On the negative side, there is no guarantee about the quality of the answers as people of very different background, knowledge, and with different moti- vation contribute answers to a given question. Unlike traditional question answering (QA), in CQA answering takes the form of commenting in a forum. Thus, many comments are only loosely connected to the original question, and some are not answers at all, but are rather interactions be- tween users. As question-comment threads can get quite long, finding good answers in a thread can be time- consuming. This has triggered research in trying to automatically determine which answers might be good and which ones are likely to be bad or ir- relevant. One early work going in this direction is that of Qu and Liu (2011), who tried to determine whether a question is “solved” or not, given its as- sociated thread of comments. As a first step in the process, they performed a comment-level classi- fication, considering four classes: problem, solu- tion, good feedback, and bad feedback. More recently, the shared task at SemEval 2015 on Answer Selection in CQA (Nakov et al., 2015), whose benchmark datasets we will use below, tackled the task of identifying good, potentially useful, and bad comments within a thread. In that task, the top participating systems used thread- level features, in addition to the usual local fea- tures that only look at the question–answer pair. For example, …

Content truncated to 500 words