Capturing Row and Column Semantics in Transformer Based Question Answering over Tables

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1212–1224 June 6–11, 2021. ©2021 Association for Computational Linguistics 1212 Capturing Row and Column Semantics in Transformer Based Question Answering over Tables Michael Glass1, Mustafa Canim1, Alfio Gliozzo1, Saneem Chemmengath1, Vishwajeet Kumar1, Rishav Chakravarti1, Avirup Sil1, Feifei Pan2, Samarth Bharadwaj1, Nicolas Rodolfo Fauceglia1 1 IBM Research AI 2 Rensselaer Polytechnic Institute mrglass@us.ibm.com, mustafa@us.ibm.com, gliozzo@us.ibm.com, saneem.cg@in.ibm.com, vishk024@in.ibm.com, rchakravarti@us.ibm.com, avi@us.ibm.com, panf2@rpi.edu, samarth.b@in.ibm.com, nicolas.fauceglia@ibm.com Abstract Transformer based architectures are recently used for the task of answering questions over tables. In order to improve the accuracy on this task, specialized pre-training techniques have been developed and applied on millions of open-domain web tables. In this paper, we propose two novel approaches demonstrat- ing that one can achieve superior performance on table QA task without even using any of these specialized pre-training techniques. The first model, called RCI interaction, leverages a transformer based architecture that indepen- dently classifies rows and columns to iden- tify relevant cells. While this model yields extremely high accuracy at finding cell val- ues on recent benchmarks, a second model we propose, called RCI representation, pro- vides a significant efficiency advantage for on- line QA systems over tables by materializ- ing embeddings for existing tables. Experi- ments on recent benchmarks prove that the pro- posed methods can effectively locate cell val- ues on tables (up to ∼98% Hit@1 accuracy on WikiSQL lookup questions). Also, the inter- action model outperforms the state-of-the-art transformer based approaches, pre-trained on very large table corpora (TAPAS and TABERT), achieving ∼3.4% and ∼18.86% additional pre- cision improvement on the standard WikiSQL benchmark1. 1 Introduction Tabular data format is a commonly used layout in domain specific enterprise documents as well as open domain webpages to store structured informa- tion in a compact form (Pasupat and Liang, 2015; Canim et al., 2019). In order to make use of these resources, many techniques have been proposed for the retrieval of tables (Cafarella et al., 2008; Zhang and Balog, 2018; Venetis et al., 2011; Shraga et al., 2020; Sun et al., 2016). Given a large corpus of 1The source code and the models we built are available at https://github.com/IBM/row-column-intersection. documents, the goal in these studies is to retrieve top-k relevant tables based on given keyword(s). The user is then expected to skim through these tables and locate the relevant cell values which is a tedious and time consuming task. More recently, popular search engines made significant improve- ment in understanding natural language questions and finding the answers within passages, owing to the developments in transformer based machine reading comprehension (MRC) systems (Rajpurkar et al., 2016, 2018; Kwiatkowski et al., 2019; Pan et al., 2019; Alberti et al., 2019a). One natural ex- tension of these systems is to answer questions over tables. These questions are broadly classified into two types: Lookup and Aggregation. Lookup ques- tions require returning exact strings from tables such as cell values whereas Aggregation questions are executed by …

Content truncated to 500 words