VIMQA: A Vietnamese Dataset for Advanced Reasoning and Explainable Multi-hop Question Answering

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6521–6529 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6521 VIMQA: A Vietnamese Dataset for Advanced Reasoning and Explainable Multi-hop Question Answering Nguyen-Khang Le1,⋆, Dieu-Hien Nguyen1,⋆, Tung Le1,2,3, Minh Le Nguyen1 1Japan Advanced Institute of Science and Technology, Ishikawa, Japan 2Faculty of Information Technology, University of Science, Ho Chi Minh city, Vietnam 3Vietnam National University, Ho Chi Minh city, Vietnam {lnkhang, ndhien}@jaist.ac.jp; lttung@fit.hcmus.edu.vn; nguyenml@jaist.ac.jp Abstract Vietnamese is the native language of over 98 million people in the world. However, existing Vietnamese Question Answering (QA) datasets do not explore the model’s ability to perform advanced reasoning and provide evidence to explain the answer. We introduce VIMQA, a new Vietnamese dataset with over 10,000 Wikipedia-based multi-hop question-answer pairs. The dataset is human-generated and has four main features: (1) The questions require advanced reasoning over multiple paragraphs. (2) Sentence-level supporting facts are provided, enabling the QA model to reason and explain the answer. (3) The dataset offers various types of reasoning to test the model’s ability to reason and extract relevant proof. (4) The dataset is in Vietnamese, a low-resource language. We also conduct experiments on our dataset using state-of-the-art Multilingual single-hop and multi-hop QA methods. The results suggest that our dataset is challenging for existing methods, and there is room for improvement in Vietnamese QA systems. In addition, we propose a general process for data creation and publish a framework for creating multilingual multi-hop QA datasets. The dataset and framework are publicly available to encourage further research in Vietnamese QA systems. Keywords: Multi-hop Question Answering, Dataset, Machine Reading Comprehension 1. Introduction Question Answering (QA) task is one of the essen- tial tasks in Natural Language Processing (NLP). In particular, most popular QA systems digest a ques- tion and contexts to reveal a correct answer. QA mod- els play a vital role in a wide range of NLP applica- tions such as search engines, intelligent agents, chat- bots. Traditionally, most popular datasets are designed to evaluate the ability of the systems to understand and answer an input question within a single docu- ment. In this type of QA task, the answer is only re- lated to a specific context such as a keyword and a sentence, which is referred to as single-hop reason- ing. To this end, there are some datasets in the English language. Particularly, Rajpurkar et al. (2016) pro- posed SQuAD, one of the large-scale span-extraction QA datasets where questions can be answered by only a single paragraph. Other single-hop datasets, TriviaQA (Joshi et al., 2017), and SearchQA (Dunn et al., 2017), have a more challenging setting where contexts are constructed from multiple documents retrieved given existing question-answer pairs. In these above datasets, complex reasoning over the coordinating context is not adequately addressed, which essentially acquires new kinds of QA tasks to accentuate the multiple associa- tions in text understanding. Accordingly, Multi-hop QA is highly essential and challenging as a potential solution in …

Content truncated to 500 words