FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6147–6152 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6147 FinMath: Injecting a Tree-structured Solver for Question Answering over Financial Reports Chenying Li†, Wenbo Ye‡, Yilun Zhao∗ †Northeastern University ‡Zhejiang University ∗Yale University †li.chenyin@northeastern.edu ‡wenbo.17@intl.zju.edu.cn Abstract Answering questions over financial reports containing both tabular and textual data (hybrid data) is challenging as it requires models to select information from financial reports and perform complex quantitative analyses. Although current models have demonstrated a solid capability to solve simple questions, they struggle with complex questions that require a multiple-step numerical reasoning process. This paper proposes a new framework named FinMath, which improves the model’s numerical reasoning capacity by injecting a tree-structured neural model to perform multi-step numerical reasoning processes. Specifi- cally, in the first phase, FinMath extracts supporting evidence from the financial reports given the question. And in the second phase, a tree-structured neural model is applied to generate a tree expression in a top-down recursive way. Experiments on the TAT-QA dataset show that FinMath improves the previous best result by 8.5% absolute for Exact Match (EM) score (50.1% to 58.6%) and 6.1% absolute for numeracy-focused F1 score (58.0% to 64.1%). Keywords: Financial NLP, Question Answering, Numerical Reasoning, Math Word Problems Solving 1. Introduction The sheer volume of financial statements and tables makes it difficult and time-consuming for humans to access and analyze financial reports. Robust numerical reasoning over hybrid data combining both tabular and textual content faces unique challenges in this domain. TAT-QA dataset (Zhu et al., 2021) focuses on questions that require numerical reasoning over financial report pages containing both paragraphs and tables. As the example shown in Figure 1, the question “What was the percentage change in gaming between 2018 and 2019?” requires the QA system to analyze the given paragraphs and tables, locate relevant cells in the tab- ular content and then perform subtraction and division operations to get the final answer. However, current best model over TAT-QA dataset, named TAGOP (Zhu et al., 2021), can only perform symbolic reasoning with a single type of pre-defined aggregation operators (e.g. change ratio, division), and might fail to answer complex questions requir- ing multi-step reasoning. To address these shortcom- ings, we present a new framework called FinMath, which can perform arbitrary steps of numerical reason- ing given the arithmetic questions. Motivated by the re- cent works in the task of Math Word Problems (MWP) solving (Xie and Sun, 2019a; Li et al., 2020; Shen and Jin, 2020), a tree-structured neural model is applied in the FinMath framework. Specifically, for those arith- metic questions in TAT-QA, after extracting the sup- porting evidence, the tree-structured neural model uses top-down goal decomposition and bottom-up subtree embedding construction to directly predict the expres- sion tree from questions and extracted evidence. Then the expression tree is executed to get the final answer. The main contribution of this work can be summarized as follows: • We propose a …

Content truncated to 500 words