Enhancing Financial Table and Text Question Answering with Tabular Graph and Numerical Reasoning

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 991–1000 November 20–23, 2022. ©2022 Association for Computational Linguistics 991 Enhancing Financial Table and Text Question Answering with Tabular Graph and Numerical Reasoning Rungsiman Nararatwong1, Natthawut Kertkeidkachorn2, Ryutaro Ichise3,1 1National Institute of Advanced Industrial Science and Technology 2Japan Advanced Institute of Science and Technology 3Tokyo Institute of Technology r.nararatwong@aist.go.jp, natt@jaist.ac.jp ichise@iee.e.titech.ac.jp Abstract Typical financial documents consist of tables, texts, and numbers. Given sufficient training data, large language models (LM) can learn the tabular structures and perform numerical rea- soning well in question answering (QA). How- ever, their performances fall significantly when data and computational resources are limited. This study improves this performance drop by infusing explicit tabular structures through a graph neural network (GNN). We proposed a model developed from the baseline of a fi- nancial QA dataset named TAT-QA. The base- line model, TagOp, consists of answer span (evidence) extraction and numerical reasoning modules. As our main contributions, we intro- duced two components to the model: a GNN- based evidence extraction module for tables and an improved numerical reasoning module. The latter provides a solution to TagOp’s arith- metic calculation problem specific to opera- tions requiring number ordering, such as sub- traction and division, which account for a large portion of numerical reasoning. Our evaluation shows that the graph module has the advantage in low-resource settings, while the improved numerical reasoning significantly outperforms the baseline model. 1 Introduction Working with tables and numerical reasoning is es- sential to understanding financial documents. How- ever, off-the-shelf pre-trained LMs generally do not understand tables and numbers. Previous QA studies on tabular data either add specialized com- ponents to LMs then finetune or modify the LMs’ architecture and pre-train with tables. The issue with these approaches is that they are not flexible to hybrid table-text data. Yet, and crucially, the model must be able to handle both data types, or it will fail to capture all the information in the documents. In 2021, (Zhu et al., 2021) introduced TAT-QA, a dataset with the abovementioned challenges. It is a collection of financial reports with questions, some requiring arithmetic operation – as part of nu- merical reasoning – on the evidence extracted from the table, text, or both. The authors also published a model named TagOp, an LM with multiple clas- sification heads for table and text-based evidence extraction and numerical reasoning. The model combines table and text as an input, performs evi- dence extraction, then applies numerical operations if needed. Our experimentation with TagOp led to two proposed components presented in this study. The first component stems from how TagOp han- dles tables. The model takes a flattened table – a sequential concatenation of table cells – as an input without additional tabular structure information. Given sufficiently large training data, the model can learn the structure well by itself. However, it appears to struggle …

Content truncated to 500 words