Predicting Helpful Posts in Open-Ended Discussion Forums: A Neural Architecture

Predicting Helpful Posts in Open-Ended Discussion Forums: A Neural Architecture Kishaloy Halder1,2 Min-Yen Kan1,2 1School of Computing, National University of Singapore 2Institute for Application of Learning Science and Educational Technology, National University of Singapore {kishaloy, kanmy, sugiyama}@comp.nus.edu.sg Kazunari Sugiyama1 Abstract Users participate in online discussion forums to learn from others and share their knowledge with the community. They often start a thread with a question or by sharing their new find- ings on a certain topic. Unlike in Commu- nity Question Answering, where questions are mostly factoid based, we find that the threads in a forum are often open-ended (e.g., asking for recommendations from others) without a definitive correct answer. We thus address the task of identifying helpful posts in a forum thread to help users comprehend long-running discussion threads, which often contain repeti- tive or irrelevant posts. We propose a recurrent neural network based architecture to model (i) the relevance of a post regarding the original post starting the thread, and (ii) the novelty it brings to the discussion, compared to the pre- vious posts in the thread. Experimental results on five different types of online forum datasets show that our model significantly outperforms the state-of-the-art neural network models for text classification. 1 Introduction Online discussion forums are widely used in many domains such as in generic web con- tent1, e-health2, Massive Open Online Courses (MOOCs)3, and e-commerce, among others. Users participate in these forums to gain knowl- edge from the collective wisdom of the commu- nity. Typically, users start a discussion thread by posting a question or asking others for opinions on a topic. Others then reply to threads relevant to their interests. Importantly, as these forums are indexed by search engines, they need to be dis- coverable by a wider audience — apart from just 1https://www.reddit.com/ 2https://www.healthboards.com/boards 3https://www.coursera.org registered users — by enabling threads to be found in response to queries. Due to the open nature of the forums and the various expertise level of users, the posts in the discussion threads vary in helpfulness. To address this, some websites provide ac- tions for users to signal this, as in “Upvote” (reddit, stackoverflow) and “Highlight” (coursera). Such feedback is helpful for iden- tifying important posts among the many. Such feedback rarely comes immediately following new post creation, affecting their visibility to the users (Singh et al., 2017). We can devise tech- nology to proactively identify such helpful posts as they arrive, in a helpfulness prediction task, en- abling users to efficiently assess relevance. We observe that there is a key structural dif- ference between online discussion forums and Community Question Answering (CQA) web- sites. Figure 1 shows the distribution of normal- ized helpful votes for the top-5 posts across a popular discussion forum (reddit), and a CQA website (stackoverflow4). In CQA, the vote distribution decays exponentially, indicating that usually there is a single correct answer with the largest number of votes (Omari et al., 2016). In contrast, votes for less helpful posts in discussion forums decay at …

Content truncated to 500 words