QCRI: Answer Selection for Community Question Answering - Experiments for Arabic and English

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 203–209, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics QCRI: Answer Selection for Community Question Answering – Experiments for Arabic and English Massimo Nicosia1, Simone Filice2, Alberto Barr´on-Cede˜no2, Iman Saleh3, Hamdy Mubarak2, Wei Gao2, Preslav Nakov2, Giovanni Da San Martino2, Alessandro Moschitti2, Kareem Darwish2, Llu´ıs M`arquez2, Shafiq Joty2 and Walid Magdy2 1 University of Trento 2 Qatar Computing Research Institute 3 Cairo University massimo.nicosia@unitn.it {sfilice,albarron,hmubarak,wgao,pnakov,gmartino}@qf.org.qa {amoschitti,kdarwish,lmarquez,sjoty,wmagdy}@qf.org.qa iman.saleh@fci-cu.edu.eg Abstract This paper describes QCRI’s participation in SemEval-2015 Task 3 “Answer Selection in Community Question Answering”, which tar- geted real-life Web forums, and was offered in both Arabic and English. We apply a super- vised machine learning approach considering a manifold of features including among others word n-grams, text similarity, sentiment anal- ysis, the presence of specific words, and the context of a comment. Our approach was the best performing one in the Arabic subtask and the third best in the two English subtasks. 1 Introduction SemEval-2015 Task 3 “Answer Selection in Com- munity Question Answering” challenged the partici- pants to automatically predict the appropriateness of the answers in a community question answering set- ting (M`arquez et al., 2015). Given a question q ∈ Q asked by user uq and a set of comments C, the main task was to determine whether a comment c ∈ C offered a suitable answer to q or not. In the case of Arabic, the questions were ex- tracted from Fatwa, a community question an- swering website about Islam.1 Each question in- cludes five comments, provided by scholars on the topic, each of which has to be automatically la- beled as (i) DIRECT : a direct answer to the ques- tion; (ii) RELATED : not a direct answer to the ques- tion but with information related to the topic; and (iii) IRRELEVANT : an answer to another question, not related to the topic. This is subtask A, Arabic. 1http://fatwa.islamweb.net In the case of English, the dataset was extracted from Qatar Living, a forum for people to pose ques- tions on multiple aspects of daily life in Qatar.2 Unlike Fatwa, the questions and comments in this dataset come from regular users, making them sig- nificantly more varied, informal, open, and noisy. In this case, the input to the system consists of a ques- tion and a variable number of comments, each of which is to be labeled as (i) GOOD : the comment is definitively relevant; (ii) POTENTIAL : the com- ment is potentially useful; and (iii) BAD : the com- ment is irrelevant (e.g., it is part of a dialogue, unre- lated to the topic, or it is written in a language other than English). This is subtask A, English. Additionally, a subset of the questions required a YES /NO answer, and there was another subtask for them, which asked to determine whether the over- all answer to the question, according to the evidence provided by the comments, is (i) YES , (ii) …

Content truncated to 500 words