Incorporating External Knowledge to Enhance Tabular Reasoning

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2799–2809 June 6–11, 2021. ©2021 Association for Computational Linguistics 2799 Incorporating External Knowledge to Enhance Tabular Reasoning J. Neeraja∗ IIT Guwahati jneeraja@iitg.ac.in Vivek Gupta∗ University of Utah vgupta@cs.utah.edu Vivek Srikumar University of Utah svivek@cs.utah.edu Abstract Reasoning about tabular information presents unique challenges to modern NLP approaches which largely rely on pre-trained contextual- ized embeddings of text. In this paper, we study these challenges through the problem of tabular natural language inference. We pro- pose easy and effective modifications to how information is presented to a model for this task. We show via systematic experiments that these strategies substantially improve tabular inference performance. 1 Introduction Natural Language Inference (NLI) is the task of de- termining if a hypothesis sentence can be inferred as true, false, or undetermined given a premise sen- tence (Dagan et al., 2013). Contextual sentence embeddings such as BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), applied to large datasets such as SNLI (Bowman et al., 2015) and MultiNLI (Williams et al., 2018), have led to near- human performance of NLI systems. In this paper, we study the harder problem of reasoning about tabular premises, as instantiated in datasets such as TabFact (Chen et al., 2019) and InfoTabS (Gupta et al., 2020). This problem is simi- lar to standard NLI, but the premises are Wikipedia tables rather than sentences. Models similar to the best ones for the standard NLI datasets struggle with tabular inference. Using the InfoTabS dataset as an example, we present a focused study that investigates (a) the poor performance of existing models, (b) connections to information deficiency in the tabular premises, and, (c) simple yet effective mitigations for these problems. We use the table and hypotheses in Figure 1 as a running example through this paper, and re- ∗*The first two authors contributed equally to the work. The first author was a remote intern at University of Utah during the work. New York Stock Exchange Type Stock exchange Location New York City, New York, U.S. Founded May 17, 1792; 226 years ago Currency United States dollar No. of listings 2,400 Volume US$20.161 trillion (2011) H1: NYSE has fewer than 3,000 stocks listed. H2: Over 2,500 stocks are listed in the NYSE. H3: S&P 500 stock trading volume is over $10 trillion. Figure 1: A tabular premise example. The hypotheses H1 is entailed by it, H2 is a contradiction and H3 is neutral i.e. neither entailed nor contradictory. fer to the left column as its keys.1 Tabular infer- ence is challenging for several reasons: (a) Poor table representation: The table does not explicitly state the relationship between the keys and values. (b) Missing implicit lexical knowledge due to lim- ited training data: This affects interpreting words like ‘fewer’, and ‘over’ in H1 and H2 respectively. (c) Presence of distracting information: All keys except No. of listings are unrelated to the hypothe- ses H1 …

Content truncated to 500 words