VectorSLU: A Continuous Word Vector Approach to Answer Selection in Community Question Answering Systems

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 282–287, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics VectorSLU: A Continuous Word Vector Approach to Answer Selection in Community Question Answering Systems Yonatan Belinkov, Mitra Mohtarami, Scott Cyphers, James Glass MIT Computer Science and Artificial Intelligence Laboratory, Cambridge, MA 02139, USA {belinkov, mitra, cyphers, glass}@csail.mit.edu Abstract Continuous word and phrase vectors have proven useful in a number of NLP tasks. Here we describe our experience using them as a source of features for the SemEval-2015 task 3, consisting of two community question an- swering subtasks: Answer Selection for cate- gorizing answers as potential, good, and bad with regards to their corresponding questions; and YES/NO inference for predicting a yes, no, or unsure response to a YES/NO question us- ing all of its good answers. Our system ranked 6th and 1st in the English answer selection and YES/NO inference subtasks respectively, and 2nd in the Arabic answer selection subtask. 1 Introduction Continuous word and phrase vectors, in which sim- ilar words and phrases are associated with similar vectors, have been useful in many NLP tasks (Al- Rfou et al., 2013; Bansal et al., 2014; Bowman et al., 2014; Boyd-Graber et al., 2012; Chen and Rud- nicky, 2014; Guo et al., 2014; Iyyer et al., 2014; Levy and Goldberg, 2014; Mikolov et al., 2013c). To evaluate the effectiveness of continuous vector representations for Community question answering (CQA), we focused on using simple features derived from vector similarity as input to a multi-class linear SVM classifier. Our approach is language indepen- dent and was evaluated on both English and Arabic. Most of the vectors we use are domain-independent. CQA services provide forums for users to ask or answer questions on any topic, resulting in high vari- ance answer quality (M`arquez et al., 2015). Search- ing for good answers among the many responses can be time-consuming for participants. This is illus- trated by the following example of a question and subsequent answers. Q: Can I obtain Driving License my QID is written Employee? A1: the word employee is a general term that refers to all the staff in your company ... you are all considered employees of your company A2: your qid should specify what is the actual pro- fession you have. I think for me, your chances to have a drivers license is low. A3: his asking if he can obtain. means he have the driver license. Answer selection aims to automatically catego- rize answers as: good if they completely answer the question, potential if they contain useful information about the question but do not completely answer it, and bad if irrelevant to the question. In the example, answers A1, A2, and A3 are respectively classified as potential, good, and bad. The Arabic answer se- lection task uses the labels direct, related, and irrel- evant. YES/NO inference infers a yes, no, or unsure response to a question through its good answers, which might not explicitly contain yes or …

Content truncated to 500 words