Task-Driven and Experience-Based Question Answering Corpus for In-Home Robot Application in the House3D Virtual Environment

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6232–6239 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6232 Task-Driven and Experience-Based Question Answering Corpus for In-Home Robot Application in the House3D Virtual Environment Zhuoqun Xu1, Liubo Ouyang1, Yang Liu2 1Hunan University, 2Samsung Research Center Beijing {zhuoqunxu, oylb}@hnu.edu.cn yang9004.liu@samsung.com Abstract At present, more and more work has begun to pay attention to the long-term housekeeping robot scene. Naturally, we wonder whether the robot can answer the questions raised by the owner according to the actual situation at home. These questions usually do not have a clear text context, are directly related to the actual scene, and it is difficult to find the answer from the general knowledge base (such as Wikipedia). Therefore, the experience accumulated from the task seems to be a more natural choice. We present a corpus called TEQA (task-driven and experience-based question answering) in the long-term household task. Based on a popular in-house virtual environment (AI2-THOR) and agent task experiences of ALFRED, we design six types of questions along with answering including 24 question templates, 37 answer templates, and nearly 10k different question answering pairs. Our corpus aims at investigating the ability of task experience understanding of agents for the daily question answering scenario on the ALFRED dataset. Keywords: grounded QA corpus, interactive environment, task experience 1. Introduction Question answering (QA) is a very important way to research language communication, and it is also an im- portant part of natural language processing (NLP) (Yih et al., 2015; Wijmans et al., 2019) . Meanwhile, we no- tice that the trend of introducing the grounded language learning methodology is getting more attention (Wu et al., 2017; Nishida et al., 2019; Castro et al., 2020). The setting of grounded language learning is closer to the scene where natural language occurs, better presents the context where QA happens, and contains physical mapping (non-language). Assuming two men talk in a house, a default context of their dialog is the surround- ing physical environment (their house), which is hardly described by the pure text. Environmental settings have emerged one after another, especially interactive envi- ronments for natural language research. Notably, AL- FRED (Action Learning From Realistic Environments and Directives) (Shridhar et al., 2020), a benchmark for learning a mapping from natural language instructions to sequences of actions for domestic tasks.Naturally, we wonder whether the robot can answer the questions raised by the owner according to the actual situation at home. These questions usually do not have a clear text context (different from the current popular setting of reading comprehension), are directly related to the ac- tual scene, and it is difficult to find the answer from the general knowledge base (such as Wikipedia). There- fore, the experience accumulated from the task seems to be a more natural choice. For instance, the agent performs a task ”clean the let- tuce” in a kitchen scene. It will go through a series of actions: pick up, walk, …

Content truncated to 500 words