Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning

Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning Vivek Gupta1, Riyaz A. Bhat2, Atreya Ghosal3, Manish Shrivastava3, Maneesh Singh2, Vivek Srikumar1 1University of Utah, USA, 2Verisk Inc., India, 3IIIT-Hyderabad, India {vgupta, svivek}@cs.utah.edu, {riyaz.bhat, maneesh.singh}@verisk.com, {atreyee.ghosal@research, m.shrivastava}@iiit.ac.in Abstract Neural models command state-of-the-art per- formance across NLP tasks, including ones involving ‘‘reasoning’’. Models claiming to reason about the evidence presented to them should attend to the correct parts of the input while avoiding spurious patterns therein, be self-consistent in their predictions across in- puts, and be immune to biases derived from their pre-training in a nuanced, context- sensitive fashion. Do the prevalent *BERT- family of models do so? In this paper, we study this question using the problem of rea- soning on tabular data. Tabular inputs are especially well-suited for the study—they ad- mit systematic probes targeting the properties listed above. Our experiments demonstrate that a RoBERTa-based model, representative of the current state-of-the-art, fails at reason- ing on the following counts: it (a) ignores relevant parts of the evidence, (b) is over- sensitive to annotation artifacts, and (c) relies on the knowledge encoded in the pre-trained language model rather than the evidence pre- sented in its tabular inputs. Finally, through inoculation experiments, we show that fine- tuning the model on perturbed data does not help it overcome the above challenges. 1 Introduction The problem of understanding tabular or semi- structured data is a challenge for modern NLP. Recently, Chen et al. (2020b) and Gupta et al. (2020) have framed this problem as a natural language inference question (NLI, Dagan et al., 2013; Bowman et al., 2015, inter alia) via the TabFact and the INFOTABS datasets, respectively. The tabular version of the NLI task seeks to de- termine whether a tabular premise entails or con- tradicts a textual hypothesis, or is unrelated to it. One strategy for such tabular reasoning tasks relies on the successes of contextualized represen- tations (e.g., Devlin et al., 2019; Liu et al., 2019b) for the sentential version of the problem. Tables are flattened into artificial sentences using heuris- tics to be processed by these models. Surprisingly, even this na¨ıve strategy leads to high predictive accuracy, as shown not only by the introductory papers but also by related lines of recent work (e.g., Eisenschlos et al., 2020; Yin et al., 2020). In this paper, we ask: Do these seemingly ac- curate models for tabular inference effectively use and reason about their semi-structured in- puts? While ‘‘reasoning’’ can take varied forms, a model that claims to do so should at least ground its outputs on the evidence provided in its inputs. Concretely, we argue that such a model should (a) be self-consistent in its predictions across con- trolled variants of the input, (b) use the evidence presented to it, and the right parts thereof, and, (c) avoid being biased against the given evi- dence by knowledge encoded in the pre-trained embeddings. Corresponding to these three properties, we identify three dimensions on which to evaluate …

Content truncated to 500 words