QU-IR at SemEval 2016 Task 3: Learning to Rank on Arabic Community Question Answering Forums with Word Embedding

Proceedings of SemEval-2016, pages 866–871, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics QU-IR at SemEval 2016 Task 3: Learning to Rank on Arabic Community Question Answering Forums with Word Embedding Rana Malhas and Marwan Torki and Tamer Elsayed Qatar University Department of Computer Science & Engineering Doha, Qatar {rana.malhas,mtorki,telsayed}@qu.edu.qa Abstract Resorting to community question answer- ing (CQA) websites for finding answers has gained momentum in the past decade with the explosive rate at which social media has been proliferating. With many ques- tions left unanswered on those websites, auto- matic and smart question answering systems have seen light. One of the main objectives of such systems is to harness the plethora of existing answered questions; hence trans- forming the problem to finding good an- swers to newly posed questions from simi- lar previously-answered ones. As SemEval 2016 Task 3 “Community Question Answer- ing” has focused on this problem, we have par- ticipated in the Arabic Subtask. Our system has adopted a supervised learning approach in which a learning-to-rank model is trained over data (questions and answers) extracted from Arabic CQA forums using word2vec features generated from that data. Our primary sub- mission achieved a 29.7% improvement over the MAP score of the baseline. Post submis- sion experiments were further conducted to in- tegrate variations of the word2vec features to our system. Integrating covariance word em- bedding features has raised the the improve- ment over the baseline to 37.9%. 1 Introduction The ubiquitous presence of community question answering (CQA) websites has motivated research in the direction of building automatic question answering (QA) systems that can benefit from previously-answered questions to answer newly- posed ones (Shtok et al., 2012). A core functionality of such systems is their ability to effectively rank previously-suggested answers with respect to their degree/probability of relevance to a posted question. The ranking functionality is vital to push away irrel- evant and low quality answers, which is common- place in CQA as they are generally open with no restrictions on who can post or answer questions. Question: ام يھ يلاودلا امو يھ اھبابسا Candidate question-answer pairs (QApairs): Q : لھ ليلاود مدقلا ضارعا دجوي يعم يلاود يف مدقلا نكلو بلطا مكنم يديوزت ضارعاب يلاودلا لكاشملاو يتلا اھببسي A :ضرعلا رثكلأا اعويش وھ ھظرو ةدرولأا ةخفتنم ىلع ةھجاو قاسلا صيخشتلااو نوكي نم للاخ صحفلا يبطلا رشابملا جلاعلاو نوكيس قفو ببسلا Q :ىھام ىلاود هجردلا هثلاثلا امو اھبابسا فيكو متي هياقولا اھنم A :يلاودلا يھ هرابع نع عسوت يف هدرولاا هيولعلا هقطنملل هباصملا اھلو ريثكلا نم بابسلاا لمحلاك هنمسلاو سولجلاو هدمل هليوط متيو ادج هياقولا اھنم نع قيرط مدع فوقولا هدمل هليوط ةسراممو هضيارلا لكشب ربكا سبلو براوج هصصخم هذھل تلااحلا عنمل مقافتلا Q :لھ قورعلا هيوتلملا لكشب حضاو يف مدقلا ربتعت نم يلاودلا لھو يلاودلا اھل راطخا اذا مل متي اھجلاع A :نكمي نأ رجفنت يلاودلا تحت دلجلا ىدؤتو ىلإ تامدك تابسرتو ةيومد تاغبصتو تحت دلجلا نكميو نأ يدؤت ىلإ تاحرقت .دق لمشي جلاع يلاود نيقاسلا ددع نم ريبادتلا يتلا فدھت ىلا فيفختلا نم ةدح ضارعلأا …

Content truncated to 500 words