A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1596–1606, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1596 A Multi-Type Multi-Span Network for Reading Comprehension that Requires Discrete Reasoning Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li National University of Defense Technology, Changsha, China {huminghao09,pengyuxing,huangzhen,dsli}@nudt.edu.cn Abstract Rapid progress has been made in the field of reading comprehension and question answer- ing, where several systems have achieved hu- man parity in some simplified settings. How- ever, the performance of these models de- grades significantly when they are applied to more realistic scenarios, where answers are involved with various types, multiple text strings are correct answers, or discrete rea- soning abilities are required. In this paper, we introduce the Multi-Type Multi-Span Net- work (MTMSN), a neural reading comprehen- sion model that combines a multi-type answer predictor designed to support various answer types (e.g., span, count, negation, and arith- metic expression) with a multi-span extraction method for dynamically producing one or mul- tiple text spans. In addition, an arithmetic expression reranking mechanism is proposed to rank expression candidates for further con- firming the prediction. Experiments show that our model achieves 79.9 F1 on the DROP hid- den test set, creating new state-of-the-art re- sults. Source code1 is released to facilitate fu- ture work. 1 Introduction This paper considers the reading comprehension task in which some discrete-reasoning abilities are needed to correctly answer questions. Specifi- cally, we focus on a new reading comprehension dataset called DROP (Dua et al., 2019), which requires Discrete Reasoning Over the content of Paragraphs to obtain the final answer. Unlike pre- vious benchmarks such as CNN/DM (Hermann et al., 2015) and SQuAD (Rajpurkar et al., 2016) that have been well solved (Chen et al., 2016; Devlin et al., 2019), DROP is substantially more challenging in three ways. First, the answers to 1https://github.com/huminghao16/MTMSN the questions involve a wide range of types such as numbers, dates, or text strings. Therefore, var- ious kinds of prediction strategies are required to successfully find the answers. Second, rather than restricting the answer to be a span of text, DROP loosens the constraint so that answers may be a set of multiple text strings. Third, for questions that require discrete reasoning, a system must have a more comprehensive understanding of the context and be able to perform numerical operations such as addition, counting, or sorting. Existing approaches, when applied to this more realistic scenario, have three problems. First, to produce various answer types, Dua et al. (2019) extend previous one-type answer predic- tion (Seo et al., 2017) to multi-type prediction that supports span extraction, counting, and ad- dition/subtraction. However, they have not fully considered all potential types. Take the question “What percent are not non-families?” and the pas- sage snippet “39.9% were non-families” as an ex- ample, a negation operation is required to infer the answer. Second, previous reading comprehension models (Wang et al., 2017; Yu et …

Content truncated to 500 words