AttesTable at SemEval-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1276–1282 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1276 AttesTable at SemEval-2021 Task 9: Extending Statement Verification with Tables for Unknown Class, and Semantic Evidence Finding Harshit Varma and Aadish Jain Indian Institute of Technology Bombay Mumbai, Maharashtra, India {harshitvarma,aadishjain}@cse.iitb.ac.in Pratik Ratadiya and Abhishek Rathi vCreaTek Consulting Services Pvt Ltd Pune, Maharashtra, India {pratik.r,abhishek.r}@vcreatek.com Abstract This paper describes our approach for Task 9 of SemEval 2021: Statement Verification and Evidence Finding with Tables. We participated in both subtasks, namely statement verification and evidence finding. For the subtask of state- ment verification, we extend the TAPAS model to adapt to the ‘unknown’ class of statements by finetuning it on an augmented version of the task data. For the subtask of evidence find- ing, we finetune the DistilBERT model in a Siamese setting. 1 Introduction Tables provide a compact and structured way of pre- senting information. They are easily interpretable by humans and are widely used in scientific papers, business articles, and even in government reports. At the same time, it is easy to misinterpret tabular data and even use it maliciously to spread mis- information. Thus, systems that can verify facts from tables and locate evidence in them can go a long way in ameliorating issues related to table interpretation. Such systems can also be used for question-answering over large tables, that are dif- ficult to analyze manually, since the task of fact verification with evidence finding is closely related to that of question answering (Jobanputra, 2019). Table entailment refers to the task of finding whether a sentence is refuted or supported by a given table. Traditionally, it has been considered a binary classification task. However, there could be instances where the table is not capable enough to either refute or accept the given statement, meaning the statement context is “unknown” for the table. In this paper, we have presented a table entailment setup that extends to three classes: refuted, entailed, and unknown, as a part of the Sem-Tab-Fact task (Wang et al., 2021). The task could be divided into two parts: statement verification, and evidence find- ing. Given a table and a statement, one has to first determine whether the table entails the statement, refutes the statement, or has insufficient informa- tion about it. This forms the subtask of statement verification. If the table entails or refutes the state- ment, one would also like to know which cells in the table provided evidence for the same. This is the subtask of evidence finding, that can be formu- lated as a binary classification problem. Each cell in the table is assigned one of the ‘relevant’ or ‘ir- relevant’ labels depending on whether it provided evidence for the given statement. Recent years have seen a rise in the use of trans- fer learning in language processing (Malte and Ratadiya, 2019a) owing to their superior perfor- mance. Models based on underlying concepts like attention mechanism and transformers are …

Content truncated to 500 words