MMQA: A Multi-domain Multi-lingual Question-Answering Framework for English and Hindi

MMQA: A Multi-domain Multi-lingual Question-Answering Framework for English and Hindi Deepak Gupta, Surabhi Kumari, Asif Ekbal and Pushpak Bhattacharyya Department of Computer Science and Engineering Indian Institute of Technology Patna Patna, India-801106 {deepak.pcs16, surabhi.mtmc16, asif, pb}@iitp.ac.in Abstract In this paper, we assess the challenges for multi-domain, multi-lingual question answering, create necessary resources for benchmarking and develop a baseline model. We curate 500 articles in six different domains from the web. These articles form a comparable corpora of 250 English documents and 250 Hindi documents. From these comparable corpora, we have created 5, 495 question-answer pairs with the questions and answers, both being in English and Hindi. The question can be both factoid or short descriptive types. The answers are categorized in 6 coarse and 63 finer types. To the best of our knowledge, this is the very first attempt towards creating multi-domain, multi-lingual question answering evaluation involving English and Hindi. We develop a deep learning based model for classifying an input question into the coarse and finer categories depending upon the expected answer. Answers are extracted through similarity computation and subsequent ranking. For factoid question, we obtain an MRR value of 49.10% and for short descriptive question, we obtain a BLEU score of 41.37%. Evaluation of question classification model shows the accuracies of 90.12% and 80.30% for coarse and finer classes, respectively. Keywords: Multi-lingual Question answering, Answer extraction, Neural network, Question classification 1. Introduction Question answering (QA) is an important area with a wide range of applicability in various Natural Language Process- ing (NLP) tasks, such as information retrieval, information extraction etc. The aim of a QA system is to automatically extract/generate the answer(s) for a given question from the data repository (e.g., web, document etc.). In a QA system questions are formulated in natural languages and answers are also dealing with natural languages. Unlike search en- gine instead of extracting information, here a QA system usually focus on extracting relevant and precise answer(s). In other words, we can say QA system is the extended mod- ification in the search engine. For achieving QA system, usually three subprocesses are followed i) question classifi- cation ii) document(s)/passage(s) extraction and iii) appro- priate answer(s) extraction. Most of the existing works fo- cus on retrieving answers in the same language in which the questions are posed. However, with the rapid growth of multilingual contents on the web, it is necessary to build an automated system that retrieves information from the doc- uments written in multiple languages. Posing questions in multiple languages and retrieving an- swers accordingly is known as Multilingual Question An- swering (MQA), which has emerged as an interesting re- search area in QA. This enables the situation where the question could be in a different language from the language of documents where the answer(s) lies. This allows users to interact in their native languages, facilitating multilingual information access, which is immensely useful in a coun- try like India. MQA system can contribute to conserving the endangered languages which are losing their …

Content truncated to 500 words