ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 778–791 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 778 Disentangling Representations of Text by Masking Transformers Xiongyi Zhang Northeastern University zhang.xio@northeastern.edu Jan-Willem van de Meent University of Amsterdam Northeastern University j.vandemeent@northeastern.edu Byron C. Wallace Northeastern University b.wallace@northeastern.edu Abstract Representations from large pretrained models such as BERT encode a range of features into monolithic vectors, affording strong predictive accuracy across a range of downstream tasks. In this paper we explore whether it is possible to learn disentangled representations by iden- tifying existing subnetworks within pretrained models that encode distinct, complementary aspects. Concretely, we learn binary masks over transformer weights or hidden units to un- cover subsets of features that correlate with a specific factor of variation; this eliminates the need to train a disentangled model from scratch for a particular task. We evaluate this method with respect to its ability to disentan- gle representations of sentiment from genre in movie reviews, toxicity from dialect in Tweets, and syntax from semantics. By combining masking with magnitude pruning we find that we can identify sparse subnetworks within BERT that strongly encode particular aspects (e.g., semantics) while only weakly encoding others (e.g., syntax). Moreover, despite only learning masks, disentanglement-via-masking performs as well as — and often better than — previously proposed methods based on varia- tional autoencoders and adversarial training. 1 Introduction and Motivation Large pretrained models such as ELMo (Peters et al., 2018), BERT (Devlin et al., 2019), and XL- Net (Yang et al., 2019) have come to dominate mod- ern NLP. Such models rely on self-supervision over large datasets to learn general-purpose representa- tions of text that achieve strong predictive perfor- mance across a spectrum of downstream tasks (Liu et al., 2019). A downside of such learned represen- tations is that it is not obvious what information they encode, which hinders model robustness and interpretability. The opacity of embeddings pro- duced by models such as BERT has motivated NLP research on designing probing tasks as a means of uncovering the properties of input texts that are Mask Weight Mask Hidden * * = = W W M M h h’ h h’ Input X or Lower layers Additional layers Additional layers Figure 1: Masking weights and hidden activations in BERT. We show a linear layer with weights W, inputs h, and outputs h′. We learn a mask for each disentan- gled factor, which is either applied to the weights W or to intermediate representations h. encoded in learned representations (Rogers et al., 2020; Linzen et al., 2019; Tenney et al., 2019). In this paper we investigate whether we can un- cover disentangled representations from pretrained models. That is, rather than mapping inputs onto a single vector that captures arbitrary combinations of features, our aim is to extract a representation that factorizes into distinct, complementary proper- ties of inputs. Explicitly factorizing representations aids interpretability, in the sense that it becomes more straightforward to determine which factors of variation inform …

Content truncated to 500 words