BreakingBERT@IITK at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 327–337 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 327 BreakingBERT@IITK at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables Aditya Jindal∗ Ankur Gupta∗ Jaya Srivastava∗ Preeti Menghwani∗ Vijit Malik∗ Vishesh Kaushik∗ Ashutosh Modi Indian Institute of Technology Kanpur (IIT Kanpur) {adityaji, ankugupt, jayasri}@iitk.ac.in {priti, vijitvm, kvishesh}@iitk.ac.in ashutoshm@cse.iitk.ac.in Abstract Recently, there has been an interest in factual verification and prediction over structured data like tables and graphs. To circumvent any false news incident, it is necessary to not only model and predict over structured data efficiently but also to explain those predictions. In this pa- per, as part of the SemEval-2021 Task 9, we tackle the problem of fact verification and evi- dence finding over tabular data. There are two subtasks. Given a table and a statement/fact, subtask A determines whether the statement is inferred from the tabular data, and subtask B determines which cells in the table provide ev- idence for the former subtask. We make a com- parison of the baselines and state-of-the-art ap- proaches over the given SemTabFact dataset. We also propose a novel approach CellBERT to solve evidence finding as a form of the Nat- ural Language Inference task. We obtain a 3- way F1 score of 0.69 on subtask A and an F1 score of 0.65 on subtask B. 1 Introduction Textual Inference, also known as natural language inference (Bowman et al., 2015), plays an impor- tant role in the study of natural language under- standing and semantic representation. Due to the unprecedented amount of information generated over the internet, it becomes essential for machines to comprehend new information based on previ- ous knowledge. Recent social events like political elections and pandemic spread have also shown the need for intelligent fact-checking systems that majorly depends on textual Inference over the sci- entific data. Though Textual Inference is well explored, the current works mainly deal with unstructured Evi- dence in the form of sentences (Dagan et al., 2005). ∗ Authors equally contributed to this work. Names in alphabetical order. Verification under structured and semi-structured Evidence, such as tables, graphs, and databases, remains unexplored. Tables are ubiquitous in doc- uments and presentations for concisely conveying important information; however, Inference on struc- tured data like tables or graphs is much more diffi- cult than simple text format due to complex struc- ture and non-universal schema for the representa- tion of data. Though recently, there has been work on Tabular Inference problems ( Zhong et al., 2020; Cho et al., 2018; Sun et al., 2018; Wenhu Chen and Wang, 2020; Eisenschlos et al., 2020; Pasupat and Liang, 2015; Wang et al., 2018 ) explaining the prediction, evidence finding is still an unexplored area. Through the SemEval-2021 Task 9 (Wang et al., 2021) we have tried to solve the Tabular Inference problem over scientific tables by providing an an- swer as well as a solution to our reasoning. In other words, given the structured …

Content truncated to 500 words