ViVQA: Vietnamese Visual Question Answering

ViVQA: Vietnamese Visual Question Answering Khanh Quoc Tran, An Trong Nguyen, An Tran-Hoai Le, Kiet Van Nguyen University of Information Technology, Ho Chi Minh City, Vietnam Vietnam National University, Ho Chi Minh City, Vietnam {18520908, 18520434, 18520426}@gm.uit.edu.vn, kietnv@uit.edu.vn Abstract Visual question answering (VQA) is a hot topic that has recently drew the atten- tion of researchers from domains as di- verse as natural language processing and computer vision. In spite of the fact that numerous VQA datasets and models have been fundamentally considered in English, there are no related works in Vietnamese VQA. In this article, we have created ViVQA, a new dataset for Vietnamese Visual Question Answering consisting of 10,328 images and 15,000 question-answer pairs in Vietnamese for evaluating Viet- namese VQA models. On the ViQA dataset, we also propose a system that uses the Hierarchical Co-Attention Model and achieves Accuracy, WUPS 0.9, and WUPS 0.0 of 0.3496, 0.4513, and 0.7786, respec- tively. Our system beats the two baseline models (LSTM and BiLSTM) on the ViQA dataset. These findings are optimistic for the development of visual question- answering systems in Vietnam. Finally, we discuss future prospects for ViVQA mod- els in order to improve performance. Our dataset1 is available freely for research pur- poses. 1 Introduction Visual question answering (VQA) is a new field that has gradually gained traction and made substantial progress in recent years. VQA is also one of the potential research areas with a 1https://github.com/khanhtran0412/ViVQA combination of natural language processing and computer vision. A VQA system can extract a proper answer to a question based on an image and a question related with it. Although the task is simple for humans, it is a challenge for computers. VQA has practical applications in our life. For example, to answer inquiries and discover information, we may incorporate automated vi- sual question answering systems into the Chat- bot platform. Visual question answering sys- tems are needed for many real-world situations, including customer support, recommendations, question answering, dialogue, and customer sys- tems management. In addition, it has incred- ible capabilities for situations such as making displayers aware of essential and valuable infor- mation from their surroundings. Visual question answering systems play an es- sential role in AI applications for human life. Al- though current research works are available in English, Japanese, and a few other languages. There are no studies on visual question answer- ing in Vietnamese because of data limitations for research. For that reason, we decided to carry out this study to build a new dataset for evalu- ating Vietnamese visual question answering sys- tems. This dataset is built based on the image data source from MS COCO. Along with im- plementing the VQA model, we evaluated the performance of the models on the dataset. We described the refinements in the models we im- plemented to find the model that gives the best results with this dataset using different state- of-the-art methods such as Deeper Long Short Term Memory, Bidirectional-Long Short Term Memory, and Hierarchical Co-Attention. In this …

Content truncated to 500 words