PerCQA: Persian Community Question Answering Dataset

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6083–6092 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6083 PerCQA: Persian Community Question Answering Dataset Naghme Jamali, Yadollah Yaghoobzadeh, Heshaam Faili School of Computer Science, Institute for Research in Fundamental Sciences (IPM) School of Electrical and Computer Engineering, College of Engineering, University of Tehran School of Electrical and Computer Engineering, College of Engineering, University of Tehran Tehran, Iran naghme.jamali@ipm.ir, y.yaghoobzadeh@ut.ac.ir, hfaili@ut.ac.ir Abstract Community Question Answering (CQA) forums provide answers to many real-life questions. These forums are trendy among machine learning researchers due to their large size. Automatic answer selection, answer ranking, question retrieval, expert finding, and fact-checking are example learning tasks performed using CQA data. This paper presents PerCQA, the first Persian dataset for CQA. This dataset contains the questions and answers crawled from the most well-known Persian forum. After data acquisition, we provide rigorous annotation guidelines in an iterative process and then the annotation of question-answer pairs in SemEvalCQA format. PerCQA contains 989 questions and 21,915 annotated answers. We make PerCQA publicly available to encourage more research in Persian CQA. We also build strong benchmarks for the task of answer selection in PerCQA by using mono- and multi-lingual pre-trained language models. Keywords: community question answering, dataset, answer selection, Persian Language 1. Introduction A Community Question Answering (CQA) platform allows web users to get answers to their questions from domain experts. With the advent of websites such as Yahoo! Answers1, Stack Exchange2, and Quora3, CQA has attracted a lot of attention. CQA is consid- ered a reliable source for acquiring the required knowl- edge to solve problems that cannot be solved directly by searching on web pages. Due to the significant expansion of these forums in terms of topics and the number of users in different languages, the administra- tors’ manual checking, and verification of contents are very challenging. Therefore, automatic solutions are required primarily to direct users to the most appropri- ate answers for each question. The ability to automatically find relevant questions to reuse their existing answers (i.e., question retrieval) and searching for relevant answers among many re- sponses for a given question (i.e., answer selection) are famous tasks in CQA. Most current work in this area is conducted using publicly available datasets, such as SemEvalCQA (Nakov et al., 2015; Nakov et al., 2016a; Nakov et al., 2017), TREC QA (Wang et al., 2007), Wiki QA (Yang et al., 2015), and InsuranceQA (Feng et al., 2015). Here, we focus on datasets in the Persian language. One significant limitation for developing au- tomatic systems to manage the Persian CQA forums is the lack of such datasets in the Persian language. This affects the quality and usability of these forums. By re- searching these forums, their quality can be improved because the users are able to get the answers they desire more efficiently. In this study, we build a dataset for Persian CQA called 1https://answers.yahoo.com/ 2https://stackexchange.com/. 3https://www.quora.com/. PerCQA. It consists of …

Content truncated to 500 words