Classifying User Forum Participants: Separating the Gurus from the Hacks, and Other Tales of the Internet

Classifying User Forum Participants: Separating the Gurus from the Hacks, and Other Tales of the Internet Marco Lui and Timothy Baldwin NICTA VRL Department of Computer Science and Software Engineering University of Melbourne, VIC 3010, Australia saffsd@gmail.com, tb@ldwin.net Abstract This paper introduces a novel user clas- sification task in the context of web user forums. We present a definition of four basic user characteristics and an anno- tated dataset. We outline a series of ap- proaches for predicting user characteris- tics, utilising aggregated post features and user/thread network analysis in a super- vised learning context. Using the pro- posed feature sets, we achieve results above both a naive baseline and a bag-of- words approach, for all four of our basic user characteristics. In all cases, our best- performing classifier is statistically indis- tinct from an upper bound based on the inter-annotator agreement for the task. 1 Introduction The most natural form of communication is through dialogue, and in the Internet age this man- ifests itself via modalities such as forums and mailing lists. What these systems have in com- mon is that they are a textual representation of a threaded discourse. The Internet is full of com- munities which engage in innumerable discourses, generating massive quantities of data in the pro- cess. This data is rich in information, and with the help of computers we are able to archive it, index it, query it and retrieve it. In theory, this would allow people to take a question to an online com- munity, search its archives for the same or similar questions, follow up on the contents of prior dis- cussion and find an answer. However, in practice, search forum accessibility tends to be limited at best, prompting recent interest in information ac- cess for user forums (Cong et al., 2008; Elsas and Carbonell, 2009; Seo et al., 2009). One problem with current approaches to access- ing forum data is that they tend not to take into account the structure of the discourse itself, or other characteristics of the forum or forum partic- ipants. The bag-of-words (BOW) model common in information retrieval (IR) and text categorisa- tion discards all contextual information. However, even in IR it has long been known that much more information than simple term occurrence is avail- able. In the modern era of web search, for exam- ple, extensive use is made of link structure (Brin and Page, 1998), anchor text, document zones, and a plethora of other document (and query, click stream and user) features (Manning et al., 2008). The natural question to ask at this point is, What additional structure can we extract from web fo- rum data? Previous work has been done in ex- tracting useful information from various dimen- sions of web forums, such as the post-level struc- ture (Kim et al., 2010). One dimension that has re- ceived relatively little attention is how we can use information about the identity of the participants to extract useful information from a web forum. In this …

Content truncated to 500 words