On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation

Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 194 - 212 December 7-8, 2022 ©2022 Association for Computational Linguistics On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation Gal Patel Leshem Choshen Omri Abend School of Computer Science and Engineering The Hebrew University of Jerusalem first.last@mail.huji.ac.il Abstract We present a methodology that explores how sentence structure is reflected in neural repre- sentations of machine translation systems. We demonstrate our model-agnostic approach with the Transformer English-German translation model. We analyze neuron-level correlation of activations between paraphrases while dis- cussing the methodology challenges and the need for confound analysis to isolate the ef- fects of shallow cues. We find that similarity between activation patterns can be mostly ac- counted for by similarity in word choice and sentence length. Following that, we manipu- late neuron activations to control the syntactic form of the output. We show this interven- tion to be somewhat successful, indicating that deep models capture sentence-structure distinc- tions, despite finding no such indication at the neuron level. To conduct our experiments, we develop a semi-automatic method to generate meaning-preserving minimal pair paraphrases (active-passive voice and adverbial clause-noun phrase) and compile a corpus of such pairs.1 1 Introduction Understanding the roles neurons play is important for the interpretability of neural machine transla- tion (NMT) models. Finding neurons that are either invariant or sensitive to particular structural distinc- tions may explain how such structures are encoded, and validates the robustness of translation systems, which is a challenging but important problem (He et al., 2020; Freitag et al., 2020). Furthermore, understating how these encodings are used by the network may potentially enable controlling the out- put by direct manipulation of neurons. Previous work analyzing what aspects of sen- tence structure are encoded in network representa- tions mostly took a probing approach or focused 1The dataset is provided with paraphrase gener- ation code: https://github.com/galpatel/ minimal-paraphrases on syntactic agreements. Works that compared ac- tivations did so either across models with identical input (Dalvi et al., 2019; Bau et al., 2019; Wu et al., 2020) or by representation words, not sentences (Antverg and Belinkov, 2021). Our novelty lies in using the same model with paraphrased input pairs, to analyze sentence structure encoding, with- out probing (c.f. §7). We derive inspiration from Computer Vision works that analyze model behav- ior under non-semantic changes to the input (Lenc and Vedaldi, 2015; Goodfellow et al., 2009). For our proposed approach, we provide a dataset of minimal paraphrases (along with a code to ex- tend it). We take two phenomena as test cases: active to passive voice and an adverbial clause to a noun phrase (see Table 1 and section §2). To compare the activation patterns of sentences that may be comprised of a different number of tokens, we aggregate tokens representations. We then measure the correlation of neuron activations between paraphrases and provide a confound anal- ysis. We find that the main contributors to strong correlation are similar positional encodings and bag-of-words overlap, …

Content truncated to 500 words