KaushikAcharya at SemEval-2021 Task 9: Candidate Generation for Fact Verification over Tables

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1271–1275 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 1271 KaushikAcharya at SemEval-2021 Task 9: Candidate Generation for Fact Verification over Tables Kaushik Acharya Philips India Ltd. / Bangalore, India acharya.kaushik@gmail.com Abstract This paper describes the system submitted in the SemEval-2021 Statement Verification and Evidence Finding with Tables task. The sys- tem relies on candidate generation for logical forms on the table based on keyword match- ing and dependency parsing on the claim state- ments. 1 Introduction Tables convey important information in a concise manner. This is true in many domains, scientific documents being one of them. Truth verification tasks in past(e.g. SemEval-2019 Fact Checking Task) have focused on written text without consid- ering the tables. The current shared task (Wang et al., 2021) focuses on tables written in English language. It requires participants to develop sys- tems to predict • veracity of textual claims (statement verifi- cation) • identify table cells forming relevant evi- dence for the claim (evidence finding) The shared task 1 comprised of two sub-tasks: 1. Subtask A: Table Statement Support 2. Subtask B: Relevant Cell Selection Subtask A is a classification problem in which the system needs to assign one of the following labels for the claim statement: • Entailed: Table supports the statement. • Refuted: Statement is contradicted by the ta- ble. • Unknown: Not enough information available in the table to assess statement’s veracity. Figure 1: Example table showing Networks across East Asia. id Claim statement 1 The n value is same for Hong Kong and Malaysia. 2 There are 9 different types country in the given table. Table 1: Statement claims of table in Figure 1 Subtask B requires finding evidence(table cells) which are minimally required to either support or refute the claim statement. This is not applicable for the statements whose veracity is unknown. These tables were sourced from scientific articles belonging to journals published by Elsevier and available on ScienceDirect. For details related to web scraping of the articles, selection criteria for choosing the tables, creating statement claims and assigning table cell evidence for the claim, please refer to the task description paper (Wang et al., 2021). An example table is shown in Fig. 1 with corresponding claims mentioned in Table 1. System described in this paper generates logical form candidates on the table data frame based on the claim statement. It executes the most probable candidate and verifies the output to check whether it matches with the one mentioned in the statement. Averaged F1 score over the tables are shown in 1https://competitions.codalab.org/ competitions/27748 1272 Table 4. Source code has been released on github 2. 2 Related Work Thorne et al. (2018) had conducted Fact Extraction and VERification (FEVER) Shared Task3 to build systems to verify claims based on evidence from Wikipedia. Similar to the current shared task, sys- tems had to label claim as Supported, Refuted or NotEnoughInfo (if there isn’t sufficient evidence to either …

Content truncated to 500 words