Multi-Hop Transformer for Document-Level Machine Translation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 325–337 June 6–11, 2021. ©2021 Association for Computational Linguistics 325 SGL: Speaking the Graph Languages of Semantic Parsing via Multilingual Translation Luigi Procopio Rocco Tripodi Sapienza NLP GroupRoberto Navigli Department of Computer Science, Sapienza University of Rome {procopio,tripodi,navigli}@di.uniroma1.it Abstract Graph-based semantic parsing aims to repre- sent textual meaning through directed graphs. As one of the most promising general-purpose meaning representations, these structures and their parsing have gained a significant interest momentum during recent years, with several di- verse formalisms being proposed. Yet, owing to this very heterogeneity, most of the research effort has focused mainly on solutions specific to a given formalism. In this work, instead, we reframe semantic parsing towards multi- ple formalisms as Multilingual Neural Ma- chine Translation (MNMT), and propose SGL, a many-to-many seq2seq architecture trained with an MNMT objective. Backed by several experiments, we show that this framework is indeed effective once the learning procedure is enhanced with large parallel corpora com- ing from Machine Translation: we report com- petitive performances on AMR and UCCA pars- ing, especially once paired with pre-trained ar- chitectures. Furthermore, we find that mod- els trained under this configuration scale re- markably well to tasks such as cross-lingual AMR parsing: SGL outperforms all its com- petitors by a large margin without even explic- itly seeing non-English to AMR examples at training time and, once these examples are in- cluded as well, sets an unprecedented state of the art in this task. We release our code and our models for research purposes at https: //github.com/SapienzaNLP/sgl. 1 Introduction Being able to associate natural language text with well-defined and machine-actionable meaning rep- resentations, i.e. the task of semantic parsing (SP), is one of the holy grails in Natural Lan- guage Processing (NLP) and Understanding (Nav- igli, 2018). Considering how a breakthrough in this direction would empower NLP systems to ex- plictly make sense of natural language, the ever- growing interest semantic parsing has been receiv- ing really comes as no surprise. Graph-based formalisms such as Abstract Meaning Represen- tation (Banarescu et al., 2013, AMR), Elemen- tary Dependency Structures (Oepen and Lønning, 2006, EDS), Prague Tectogrammatical Graphs (Ha- jiˇc et al., 2012, PTG), Universal Conceptual Cog- nitive Annotation (Abend and Rappoport, 2013, UCCA), inter alia, are emerging as the de facto stan- dard for general-purpose meaning representations and have shown potential in Machine Translation (Song et al., 2019), Text Summarization (Hardy and Vlachos, 2018), Human-Robot Interaction (Bonial et al., 2020), and as evaluation metrics (Sulem et al., 2018; Xu et al., 2020b). These formalisms propose encoding meaning through directed graphs, how- ever, each of them builds upon different linguistic assumptions, aims to target different objectives and, at a more practical level, assigns different functions to nodes and edges. For instance, while AMR uses nodes to encode concepts and edges to express the semantic relations between them, UCCA proposes using text tokens as terminal nodes and …

Content truncated to 500 words