Data Augmentation for Fake News Detection by Combining Seq2seq and NLI

Proceedings of Recent Advances in Natural Language Processing, pages 429–439 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_048 429 Data Augmentation for Fake News Detection by Combining Seq2seq and NLI Anna Glazkova University of Tyumen Tyumen, Russia a.v.glazkova@utmn.ru Abstract State-of-the-art data augmentation methods help improve the generalization of deep learn- ing models. However, these methods often generate examples that contradict the preserv- ing class labels. This is crucial for some nat- ural language processing tasks, such as fake news detection. In this work, we combine sequence-to-sequence and natural language in- ference models for data augmentation in the fake news detection domain using short news texts, such as tweets and news titles. This ap- proach allows us to generate new training ex- amples that do not contradict facts from the original texts. We use non-entailment probabil- ity for the original and generated texts as a loss function for a transformer-based sequence-to- sequence model. The proposed approach has demonstrated the effectiveness on three classi- fication benchmarks in fake news detection in terms of the F1-score macro and ROC AUC. Moreover, we showed that our approach retains the class label of the original text more accu- rately than other transformer-based methods. 1 Introduction The modern world provides great opportunities for news spreading. News travels fast, and it is diffi- cult to expeditiously confirm or deny its credibility. In this regard, there is evidence that the tools for detecting fake news play a crucial role in the regu- lation of information flows. Although machine learning models are widely used in fighting fake news, their performance de- pends on the size and quality of training data. Col- lection and annotation of text corpora require signif- icant time costs. As an interim solution, augmented data obtained from a small number of annotated texts can be used while training. Data augmentation (DA) is the artificial creation of training data for machine learning by transfor- mations (Bayer et al., 2022). Even though the cur- rent state-of-the-art DA methods show impressive results, they are still ill-suited for some natural language processing tasks, such as fake news de- tection. The bottleneck is non-conditional DA that contradicts the preserving class labels. Thus, the generated news seems to be untruthful. Neither rule-based nor model-based approaches guarantee the factual consistency of the original and gener- ated text. This can be a challenge for practical applications because the system will input fakes as examples of real news, and vice versa. In this paper, we propose a DA approach that enables the generation of training examples flow- ing logically from the original texts. To that end, we combine pre-trained sequence-to-sequence (seq2seq) models showing SoTA results in DA, with natural language inference (NLI) models esti- mating textual entailment information. The task of NLI is to predict an entailment relation label (out- put) given a premise-hypothesis pair (input) (Poliak et al., 2018). The contribution of this paper is two-fold: a) we built a model to augment data in the field of fake news detection by combining seq2seq and NLI. The …

Content truncated to 500 words