Extend, don’t rebuild: Phrasing conditional graph modification as autoregressive sequence labelling

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1213–1224 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1213 Extend, don’t rebuild: Phrasing conditional graph modification as autoregressive sequence labelling Leon Weber∆,2 and Samuele Garda∆ and Jannes Münchmeyer3,∆ and Ulf Leser∆ ∆ Humboldt-Universität zu Berlin, 2 Max Delbrück Center for Molecular Medicine, 3 GFZ German Research Center for Geoscience Potsdam weberple@hu-berlin, gardasam@hu-berlin.de, munchmej@gfz-potsdam.de, leser@informatik.hu-berlin.de Abstract Deriving and modifying graphs from natural language text has become a versatile basis technology for information extraction with ap- plications in many subfields, such as seman- tic parsing or knowledge graph construction. A recent work used this technique for modi- fying scene graphs (He et al., 2020), by first encoding the original graph and then generat- ing the modified one based on this encoding. In this work, we show that we can consider- ably increase performance on this problem by phrasing it as graph extension instead of graph generation. We propose the first model for the resulting graph extension problem based on autoregressive sequence labelling. On three scene graph modification data sets, this formu- lation leads to improvements in accuracy over the state-of-the-art between 13 and 26 percent- age points. Furthermore, we introduce a novel data set from the biomedical domain which has much larger linguistic variability and more complex graphs than the scene graph modifica- tion data sets. For this data set, the state-of-the art fails to generalize, while our model can pro- duce meaningful predictions. 1 Introduction Generating or modifying graphs based on natu- ral language texts is a versatile technique that has applications in different subfields of Natural Lan- guage Processing (NLP) such as dependency pars- ing (Manning and Schütze, 2001) or semantic pars- ing (Oepen et al., 2019, 2020). However, while these tasks can all be viewed as instantiations of conditional graph generation, they have been tradi- tionally addressed as distinct tasks with different data sets, models and evaluation settings. Con- trary to that, we are interested in studying the general task of generating or modifying graphs based on textual input. Specifically, we focus on the recently introduced task of conditional graph modification, in which a model is given a graph which it should modify according to natural lan- guage instructions (He et al., 2020). Their proposed method first embeds both the graph and the instruc- tions with a joint encoder into an embedding h and then rebuilds the graph using a separate generative model for graphs (You et al., 2018b) conditioned on h. While this approach achieves state-of-the-art results for the Scene Graph Modification (SGM) data sets, we identified two shortcomings of this approach: (i) The model has to newly generate also the parts of the input graph that actually should be left unmodified and (ii) the model uses a sepa- rate graph encoder in the generative decoder model, which does not share knowledge with the encoder. We propose an alternative formulation of this problem in which we model the modification as a graph extension instead …

Content truncated to 500 words