A Hierarchical Entity-Based Approach to Structuralize User Generated Content in Social Media: A Case of Yahoo! Answers

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1521–1532, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics A Hierarchical Entity-based Approach to Structuralize User Generated Content in Social Media: A Case of Yahoo! Answers Baichuan Li1,2∗, Jing Liu3∗, Chin-Yew Lin4, Irwin King1,2, and Michael R. Lyu1,2 1Shenzhen Research Institute, The Chinese University of Hong Kong, Shenzhen, China 2Department of Computer Science and Engineering The Chinese University of Hong Kong, Shatin, N.T., Hong Kong 3Harbin Institute of Technology, Harbin 150001, P.R. China 4Microsoft Research Asia, Beijing 100080, P.R. China bcli@cse.cuhk.edu.hk jliu@ir.hit.edu.cn cyl@microsoft.com {king,lyu}@cse.cuhk.edu.hk Abstract Social media like forums and microblogs have accumulated a huge amount of user generated content (UGC) containing human knowledge. Currently, most of UGC is listed as a whole or in pre-defined categories. This “list-based” approach is simple, but hinders users from browsing and learning knowledge of certain topics effectively. To address this problem, we propose a hierarchical entity-based approach for structuralizing UGC in social media. By using a large-scale entity repository, we design a three-step framework to organize UGC in a novel hierarchical structure called “cluster entity tree (CET)”. With Yahoo! Answers as a test case, we conduct experiments and the results show the effectiveness of our frame- work in constructing CET. We further evaluate the performance of CET on UGC organiza- tion in both user and system aspects. From a user aspect, our user study demonstrates that, with CET-based structure, users perform significantly better in knowledge learning than using traditional list-based approach. From a system aspect, CET substantially boosts the performance of two information retrieval models (i.e., vector space model and query likelihood language model). 1 Introduction With the development of Web 2.0, social media websites—such as online forums, blogs, microblogs, social networks, and community ∗This work was done when the first two authors were on internship at MSRA. Table 1: Sample questions about Edinburgh 1. Where can i buy a hamburger in Edinburgh? 2. Where can I get a shawarma in Edinburgh? 3. How long does it take to drive between Glasgow and Edinburgh? 4. Whats the difference between Glasgow and Edinburgh? 5. Good hotels in London and Edinburgh? 6. Looking for nice , clean cheap hotel in Edinburgh? 7. Does anyone know of a reasonably cheap hotel in Edinburgh that is near to Niddry Street South ? 8. Who can recommend a affordable hotel in Edinburgh City Center? question answering (CQA) portals—have become the mainstream of web, where users create, share, and exchange information with each other. As a result, more and more UGC is accumulated, with social media websites retaining a huge amount of human knowledge and user experience. At present, most of UGC is organized in a list structure with extra information (e.g., category hierarchies in online forums), or without any other information. This “list-of-content” (list-based approach) is simple and straightforward, but ineffective for browsing and knowledge learning. Consider the following case: vacation in Edinburgh.a user wants to spend his He visits a CQA …

Content truncated to 500 words