JAIST: Combining multiple features for Answer Selection in Community Question Answering

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 215–219, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics JAIST: Combining multiple features for Answer Selection in Community Question Answering Quan Hung Tran1, Vu Duc Tran1, Tu Thanh Vu2, Minh Le Nguyen1, Son Bao Pham2 1Japan Advanced Institute of Science and Technology 2University of Engineering and Technology, Vietnam National University, Hanoi 1{quanth,vu.tran,nguyenml}@jaist.ac.jp 2{tuvt,sonpb}@vnu.edu.vn Abstract In this paper, we describe our system for SemEval-2015 Task 3: Answer Selection in Community Question Answering. In this task, the systems are required to identify the good or potentially good answers from the answer thread in Community Question Answering collections. Our system combines 16 features belong to 5 groups to predict answer quality. Our final model achieves the best result in sub- task A for English, both in accuracy and F1- score. 1 Introduction Nowadays, community question answering (cQA) websites like Yahoo! Answers play a crucial role in supporting people to seek desired information. Users can post their questions on these sites for find- ing help as well as personal advice. However, the quality of these answers varies greatly. Typically, only a few of the answers in an answer thread are useful to the users and it may take a lot of efforts to identify them manually. Thus, a system that auto- matically identifies answer quality is much needed. The task of identifying answer quality has been studied by many researchers in the field of Question Answering. Many methods have been proposed: web redundancy information (Magnini et al., 2002), non-textual features (Jeon et al., 2006), textual en- tailment (Wang and Neumann, 2007), syntactic fea- tures (Grundstr¨om and Nugues, 2014). However, most of these works used independent dataset and evaluation metrics; thus it is difficult to compare the results of these methods. The SEMEVAL task 3 (M`arquez et al., 2015) addresses this problem by providing a common framework to compare differ- ent methods in multiple languages. Our system incorporates a range of features: word-matching features, special component fea- tures, topic-modeling-based features, translation- based features and non-textual features to achieve the best performance in subtask A (M`arquez et al., 2015). In the remainder of the paper, we will de- scribe our system with the focus on the features. 2 System Description For extracting the features, we first preprocess the questions and the answers then build a number of models based on training data or other sources (Fig- ure 1). 2.1 Preprocessing All the questions and the answers are preprocessed through the following steps: Tokenization, POS- tagging, Syntactic parsing, Dependency parsing, Lemmatization, Stopword removal, Name-Entity recognition. These preprocessing steps are com- pleted using The Stanford CoreNLP Natural Lan- guage Processing Toolkit (Manning et al., 2014). Because of the noisy nature of community data, the syntactic parsing, dependency parsing and Name- Entity recognition steps do not produce highly ac- curate results. Thus, we rely mainly on the bag-of- word representation of text. Removing stopwords or lemmatization can alter the meaning of the text, so in …

Content truncated to 500 words