Understanding Unnatural Questions Improves Reasoning over Text

Proceedings of the 28th International Conference on Computational Linguistics, pages 4949–4955 Barcelona, Spain (Online), December 8-13, 2020 4949 Understanding Unnatural Questions Improves Reasoning over Text Xiao-Yu Guo Yuan-Fang Li Faculty of Information Technology, Monash University, Melbourne, Australia {xiaoyu.guo, yuanfang.li, gholamreza.haffari}@monash.edu Gholamreza Haffari Abstract Complex question answering (CQA) over raw text is a challenging task. A prominent approach to this task is based on the programmer-interpreter framework, where the programmer maps the question into a sequence of reasoning actions and the interpreter then executes these actions on the raw text. Learning an effective CQA model requires large amounts of human-annotated data, consisting of the ground-truth sequence of reasoning actions, which is time-consuming and expensive to collect at scale. In this paper, we address the challenge of learning a high-quality programmer (parser) by projecting natural human-generated questions into unnatural machine- generated questions which are more convenient to parse. We firstly generate synthetic (question, action sequence) pairs by a data generator, and train a semantic parser that associates synthetic questions with their corresponding action sequences. To capture the diversity when applied to natural questions, we learn a projection model to map natural questions into their most similar unnatural questions for which the parser can work well. Without any natural training data, our projection model provides high-quality action sequences for the CQA task. Experimental results show that the QA model trained exclusively with synthetic data outperforms its state-of-the-art counterpart trained on human-labeled data. 1 Introduction The complex question answering (CQA) task, which requires multi-step, discrete actions to be executed over text to obtain answers, is a challenging task. On the recently released DROP benchmark (Dua et al., 2019), the state-of-the-art method Neural Module Networks (NMNs) (Gupta et al., 2020) learns to interpret each question as a sequence of neural modules, or discrete actions, and execute them to yield the answer. However, the supervised learning of CQA models such as NMNs requires a large amount of annotated (question, action sequence) pairs, which are expensive to acquire and augment. Therefore, the label scarcity problem remains a challenge to the CQA problem. Motivated by this, we propose a projection model to alleviate the label scarcity challenge by generat- ing synthetic training data. To our knowledge, our model is the first approach using projection to solve the label scarcity problem for the CQA task. The projection model can automatically label large amounts of unlabelled questions with action sequences, so that a CQA model can be trained without natural su- pervised data. Our method is inspired by the recent “simulation-to-real” transfer approach (Tzeng et al., 2015; Marzoev et al., 2020). As the name suggests, internal knowledge in synthetic data is firstly learned in the “simulation” phase, and then transferred “to-real” circumstances with natural-language utterance using the projection model. In the “simulation” phase, we design an n-gram based generator to produce synthetic training data, i.e. (question, action sequence) pairs which constitute the synthetic dataset. We then train a semantic parser to learn the internal knowledge in this dataset. In the …

Content truncated to 500 words