RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 851–865 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 851 Memory and Knowledge Augmented Language Models for Inferring Salience in Long-Form Stories David Wilmot School of Informatics University of Edinburgh david.wilmot@ed.ac.uk Frank Keller School of Informatics University of Edinburgh keller@inf.ed.ac.uk Abstract Measuring event salience is essential in the un- derstanding of stories. This paper takes a re- cent unsupervised method for salience detec- tion derived from Barthes Cardinal Functions and theories of surprise and applies it to longer narrative forms. We improve the standard transformer language model by incorporating an external knowledgebase (derived from Re- trieval Augmented Generation) and adding a memory mechanism to enhance performance on longer works. We use a novel approach to derive salience annotation using chapter- aligned summaries from the Shmoop corpus for classic literary works. Our evaluation against this data demonstrates that our salience detection model improves performance over and above a non-knowledgebase and memory augmented language model, both of which are crucial to this improvement. 1 Introduction Forster (1985) compared a story to a wriggling worm of time that can be seen as a series of events arranged in order (see also Abbott 2008) — dinner comes after breakfast, night after day, nemesis fol- lows hubris. Not all events are of equal importance, and some far more salient than others. For exam- ple, the beginning of Dickens’ Great Expectations — Keep still, you little devil, or I’ll cut your throat! — is more salient to the story than events such as my sister had a trenchant way of cutting our bread and butter for us. Salient events in storytelling are those that drive the plot forward, change the state in the story world, as opposed to descriptive details or non-consequential activities. As such, detect- ing salience is an essential part of understanding narrative. Detecting salient events has important downstream applications such as summarisation; salient events are the core of plots and can aid sto- ryline writing and story generation; they represent essential information and are relevant to question answering. This paper builds on the work of Otake et al. (2020), who use Barthes Cardinal Functions (BCF) for unsupervised salience detection. We augment this approach with a knowledgebase (KB) and memory. Barthes Cardinal Functions (Barthes and Duisit, 1966) are hinge events that cannot be deleted without altering the story; they are the deci- sion points between alternative consequential paths. Barthes and Duisit also define catalysers, which are inconsequential events such as the bread and butter example, indices, which are descriptive, re- ferring to a character or situation, and informants, which identify time and space. These latter types can be seen as satellites around the nuclei, or fill- ing in gaps between cardinal functions. Hence to identify BCF is to identify the main skeleton of the plot. We treat the BCF events as the salient events in a story. This scheme relates in narratology with Chatman (1980) kernels and satellites model, as well …

Content truncated to 500 words