Al-Bayan: A Knowledge-based System for Arabic Answer Selection

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 226–230, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics Al-Bayan: A Knowledge-based System for Arabic Answer Selection Reham Mohamed Maha Ragab reham.mohmd@alexu.edu.eg maha.ragab@alexu.edu.eg Heba Abdelnasser Nagwa M. El-Makky heba.abdelnasser@alexu.edu.eg Marwan Torkinagwamakky@alexu.edu.eg mtorki@alexu.edu.eg Computer and Systems Engineering Department Alexandria University, Egypt Abstract This paper describes Al-Bayan team participa- tion in SemEval-2015 Task 3, Subtask A. Task 3 targets semantic solutions for answer selec- tion in community question answering systems. We propose a knowledge-based solution for answer selection of Arabic questions, special- ized for Islamic sciences. We build a Semantic Interpreter to evaluate the semantic similarity between Arabic question and answers using our Quranic ontology of concepts. Using su- pervised learning, we classify the candidate answers according to their relevance to the users questions. Results show that our system achieves 74.53% accuracy which is compara- ble to the other participating systems. 1 Introduction With the increase of the popularity of community question answering (CQA) systems, answer selection became more challenging. CQA systems are often open for public to answer any questions with no re- striction or review from field experts. This highlights the importance of developing systems that automati- cally detects the most relevant answers from the irrel- evant ones. These systems might be open-domain or closed-domain, causing a tradeoff between accuracy and generality. SemEval-2015 task 3 targets semantically oriented solutions for answer selection in community question answering data. We focus on Subtask A for the Ara- bic language which provides questions and several community answers from the Fatwa website1. The 1Fatwa is a question about the Islamic religion. goal is to classify each answer as: Direct, Related or Irrelevant. In this paper, we propose a knowledge-based an- swer selection system for Arabic. We use our Quranic ontology, enriched with Quran verses and Tafseer books, to convert each question and its candidate an- swers into weighted vectors of ontology concepts. We use these vectors to compute a semantic similar- ity score between the question and each candidate answer. We also compute a keyword matching score and feed the two scores into a decision tree classifier which predicts how much the answer is related to the question. The rest of the paper is organized as follows: Sec- tion 2 shows some of the related work to the system. Section 3 shows the details of the system architec- ture. In Section 4, we show the results of the task evaluation. Finally, we conclude the paper in Section 5. 2 Related Work Our work is related to prior work in both Quranic research and Question Answer Selection systems. (a) Quranic Research: Several studies have been made to understand the Quranic text and extract knowledge from it using computational linguistics. Saad et al. (2009) proposed a simple methodology for automatic extraction of concepts based on the Quran in order to build an ontology. In (Saad et al., 2010), they developed a framework for automated gener- ation of Islamic knowledge concrete concepts that …

Content truncated to 500 words