Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2186–2202, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2186 Understanding the Mechanics of SPIGOT: Surrogate Gradients for Latent Structure Learning Tsvetomila Mihaylova Instituto de Telecomunicac¸˜oes Instituto Superior T´ecnico Lisbon, Portugal tsvetomila.mihaylova@lx.it.pt Vlad Niculae† Informatics Institute University of Amsterdam The Netherlands v.niculae@uva.nl Andr´e F. T. Martins Instituto de Telecomunicac¸˜oes LUMLIS (Lisbon ELLIS Unit) Instituto Superior T´ecnico & Unbabel Lisbon, Portugal andre.t.martins@tecnico.ulisboa.com Abstract Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously uncov- ering linguistic insights about the data. One challenge with end-to-end training of these models is the argmax operation, which has null gradient. In this paper, we focus on surrogate gradients, a popular strategy to deal with this problem. We explore latent structure learning through the angle of pulling back the down- stream learning objective. In this paradigm, we discover a principled motivation for both the straight-through estimator (STE) as well as the recently-proposed SPIGOT—a variant of STE for structured models. Our perspec- tive leads to new algorithms in the same family. We empirically compare the known and the novel pulled-back estimators against the pop- ular alternatives, yielding new insight for prac- titioners and revealing intriguing failure cases. 1 Introduction Natural language data is rich in structure, but most of the structure is not visible at the surface. Ma- chine learning models tackling high-level language tasks would benefit from uncovering underlying structures such as trees, sequence tags, or segmen- tations. Traditionally, practitioners turn to pipeline approaches where an external, pretrained model is used to predict, e.g., syntactic structure. The benefit of this approach is that the predicted tree is readily available for inspection, but the downside is that the errors can easily propagate throughout the pipeline and require further attention (Finkel et al., 2006; Sutton and McCallum, 2005; Toutanova, 2005). In contrast, deep neural architectures tend to eschew such preprocessing, and instead learn †Work partially done while VN was at the Instituto de Telecomunicac¸˜oes, Lisbon. soft hidden representations, not easily amenable to visualization and analysis. The best of both worlds would be to model structure as a latent variable, combining the trans- parency of the pipeline approach with the end- to-end unsupervised representation learning that makes deep models appealing. Moreover, large- capacity model tend to rediscover structure from scratch (Tenney et al., 2019), so structured latent variables may reduce the required capacity. Learning with discrete, combinatorial latent vari- ables is, however, challenging, due to the inter- section of large cardinality and null gradient is- sues. For example, when learning a latent depen- dency tree, the latent parser must choose among an exponentially large set of possible trees; what’s more, the parser may only learn from gradient in- formation from the downstream task. If the highest- scoring tree is selected using an argmax operation, the gradients will be zero, preventing learning. One strategy for dealing with the null gradi- ent issue …

Content truncated to 500 words