Metadata-Aware Measures for Answer Summarization in Community Question Answering

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 760–769, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Metadata-Aware Measures for Answer Summarization in Community Question Answering Mattia Tomasoni ∗ Dept. of Information Technology Uppsala University, Uppsala, Sweden mattia.tomasoni.8371@student.uu.se Minlie Huang Dept. Computer Science and Technology Tsinghua University, Beijing 100084, China aihuang@tsinghua.edu.cn Abstract This paper presents a framework for au- tomatically processing information com- ing from community Question Answering (cQA) portals with the purpose of gen- erating a trustful, complete, relevant and succinct summary in response to a ques- tion. We exploit the metadata intrinsically present in User Generated Content (UGC) to bias automatic multi-document summa- rization techniques toward high quality in- formation. We adopt a representation of concepts alternative to n-grams and pro- pose two concept-scoring functions based on semantic overlap. Experimental re- sults on data drawn from Yahoo! An- swers demonstrate the effectiveness of our method in terms of ROUGE scores. We show that the information contained in the best answers voted by users of cQA por- tals can be successfully complemented by our method. 1 Introduction Community Question Answering (cQA) portals are an example of Social Media where the infor- mation need of a user is expressed in the form of a question for which a best answer is picked among the ones generated by other users. cQA websites are becoming an increasingly popular complement to search engines: overnight, a user can expect a human-crafted, natural language answer tailored to her specific needs. We have to be aware, though, that User Generated Content (UGC) is often re- dundant, noisy and untrustworthy (Jeon et al., ∗The research was conducted while the first author was visiting Tsinghua University. 2006; Wang et al., 2009b; Suryanto et al., 2009). Interestingly, a great amount of information is em- bedded in the metadata generated as a byprod- uct of users’ action and interaction on Social Me- dia. Much valuable information is contained in an- swers other than the chosen best one (Liu et al., 2008). Our work aims to show that such informa- tion can be successfully extracted and made avail- able by exploiting metadata to distill cQA content. To this end, we casted the problem to an instance of the query-biased multi-document summariza- tion task, where the question was seen as a query and the available answers as documents to be sum- marized. We mapped each characteristic that an ideal answer should present to a measurable prop- erty that we wished the final summary could ex- hibit: • Quality to assess trustfulness in the source, • Coverage to ensure completeness of the in- formation presented, • Relevance to keep focused on the user’s in- formation need and • Novelty to avoid redundancy. Quality of the information was assessed via Ma- chine Learning (ML) techniques under best an- swer supervision in a vector space consisting of linguistic and statistical features about the answers and their authors. Coverage was estimated by se- mantic comparison with the knowledge space …

Content truncated to 500 words