BOUN at SemEval-2021 Task 9: Text Augmentation Techniques for Fact Verification in Tabular Data

Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 431–437 Bangkok, Thailand (online), August 5–6, 2021. ©2021 Association for Computational Linguistics 431 BOUN at SemEval-2021 Task 9: Text Augmentation Techniques for Fact Verification in Tabular Data Abdullatif K¨oksal Department of Computer Engineering Bo˘gazic¸i University abdullatif.koksal@boun.edu.tr Yusuf Y¨uksel Department of Computer Engineering Bo˘gazic¸i University yusuf.yuksel@boun.edu.tr Bekir Yıldırım Department of Computer Engineering Bo˘gazic¸i University bekir.yildirim@boun.edu.tr Arzucan ¨Ozg¨ur Department of Computer Engineering Bo˘gazic¸i University arzucan.ozgur@boun.edu.tr Abstract In this paper, we present our text augmenta- tion based approach for the Table Statement Support Subtask (Phase A) of SemEval-2021 Task 9. We experiment with different text augmentation techniques such as back transla- tion and synonym swapping using Word2Vec and WordNet. We show that text augmenta- tion techniques lead to 2.5% improvement in F1 on the test set. Further, we investigate the impact of domain adaptation and joint learn- ing on fact verification in tabular data by uti- lizing the SemTabFacts and TabFact datasets. We observe that joint learning improves the F1 scores on the SemTabFacts and TabFact test sets by 3.31% and 0.77%, respectively. 1 Introduction Recognizing Textual Entailment (RTE) (Dagan et al., 2005) is one of the core NLP problems for un- derstanding the semantic relations between words and sentences, which is useful for other tasks in- cluding Question Answering (Abacha and Demner- Fushman, 2019), Text Summarization (Lloret et al., 2008), and Text Classification (Yin et al., 2019). For the RTE task, datasets of various sizes (Da- gan et al., 2005; Bowman et al., 2015) and from different domains (Romanov and Shivade, 2018) have been introduced. However, these works and datasets are solely focused on textual data without considering structured data such as tables. Recently, question answering (Iyyer et al., 2017) and textual entailment datasets (Wenhu Chen and Wang, 2020; Wang et al., 2021) for tabular data have been introduced. SemEval-2021 Task 9 addresses the problem of statement verification 1The grammatical error exists in the given dataset. Distance statistics between buildings of ancient buildings and modern buildings to the main water channel (unit: meter). Index Ancient building Modern building AVERAGE 174.095 273.917 STDEV.S 58.780 190.928 MIN 6.763 4.868 MAX 321.608 912.368 MEDIAN 173.010 243.885 Statement Label There are 2 types of building - Ancient building and Modern building. Entailed All the values of Ancient build- ing is less than Modern building except MIN value. Entailed The value of Modern building is is lesser than Ancient building in AVERAGE. 1 Refuted Figure 1: Sample table, description, and statements from SemTabFacts. (Phase A) and evidence finding (Phase B) using tables from scientific articles (Wang et al., 2021). The shared task also introduced a new dataset, namely the SemTabFacts dataset, an example from which is provided in Figure 1. The goal of Phase A (Table Statement Verification) of the shared task is to determine whether a statement is entailed, re- futed, or unknown given a table and its description (if available). For example, given the table and its description in Figure 1, the first two …

Content truncated to 500 words