A DQN-based Approach to Finding Precise Evidences for Fact Verification

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 1030–1039 August 1–6, 2021. ©2021 Association for Computational Linguistics 1030 A DQN-based Approach to Finding Precise Evidences for Fact Verification Hai Wan1, Haicheng Chen1, Jianfeng Du2,3∗, Weilin Luo1, Rongzhen Ye1 1 School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou 510006, P.R.China 2 Guangzhou Key Laboratory of Multilingual Intelligent Processing, Guangdong University of Foreign Studies, Guangzhou 510006, P.R.China 3 Pazhou Lab, Guangzhou 510330, P.R.China wanhai@mail.sysu.edu.cn, jfdu@gdufs.edu.cn, {chenhch8, luowlin3, yerzh}@mail2.sysu.edu.cn Abstract Computing precise evidences, namely mini- mal sets of sentences that support or refute a given claim, rather than larger evidences is cru- cial in fact verification (FV), since larger ev- idences may contain conflicting pieces some of which support the claim while the other refute, thereby misleading FV. Despite being important, precise evidences are rarely stud- ied by existing methods for FV. It is challeng- ing to find precise evidences due to a large search space with lots of local optimums. In- spired by the strong exploration ability of the deep Q-learning network (DQN), we propose a DQN-based approach to retrieval of precise ev- idences. In addition, to tackle the label bias on Q-values computed by DQN, we design a post- processing strategy which seeks best thresh- olds for determining the true labels of com- puted evidences. Experimental results confirm the effectiveness of DQN in computing pre- cise evidences and demonstrate improvements in achieving accurate claim verification.1 1 Introduction With the growing false information, such as fake news, political deception and online rumors, auto- matic fact-checking systems have emerged to auto- matically identify and filter this information. Fact verification (FV) is a special fact-checking task that aims to retrieve related evidences from a text corpus to verify a textual claim. Taking Figure 1 as example, an existing method for FV first retrieves related documents from the given corpus at stage 1 (namely the document re- trieval stage), then finds key sentences from the documents at stage 2 (namely the sentence selec- tion stage), and finally treats the set of key sen- tences as an evidence to verify the claim at stage ∗Corresponding author 1Source code and data are available at https:// github.com/sysulic/DQN-FV. Figure 1: The pipeline for FV on FEVER. Underlined words in blue italics given in evidence provide key in- formation to determine the truthfulness of the claim. “SUPPORTS” / “REFUTES” / “NOT ENOUGH INFO” indicates that the evidence can support / refute / is in- sufficient for supporting or refuting the claim. Both the evidence and label are output by FV. 3 (namely the claim verification stage). As can be seen in this example, it is desirable to retrieve an evidence consisting of the first two sentences only, since it does not contain unnecessary sentences to determine the truthfulness of the claim and can alle- viate human efforts to further validate the evidence. More importantly, an evidence containing unneces- sary sentences may involve conflicting …

Content truncated to 500 words