Hierarchical Recurrent Aggregative Generation for Few-Shot NLG

Findings of the Association for Computational Linguistics: ACL 2022, pages 2167 - 2181 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Hierarchical Recurrent Aggregative Generation for Few-Shot NLG Giulio Zhou Huawei Noah’s Ark Lab London, UK Gerasimos Lampouras Huawei Noah’s Ark Lab London, UK {giuliozhou, gerasimos.lampouras, ignacio.iacobacci}@huawei.com Ignacio Iacobacci Huawei Noah’s Ark Lab London, UK Abstract Large pretrained models enable transfer learn- ing to low-resource domains for language gen- eration tasks. However, previous end-to-end approaches do not account for the fact that some generation sub-tasks, specifically aggre- gation and lexicalisation, can benefit from transfer learning to different extents. To ex- ploit these varying potentials for transfer learn- ing, we propose a new hierarchical approach for few-shot and zero-shot generation. Our approach consists of a three-moduled jointly trained architecture: the first module indepen- dently lexicalises the distinct units of infor- mation in the input as sentence sub-units (e.g. phrases), the second module recurrently aggre- gates these sub-units to generate a unified in- termediate output, while the third module sub- sequently post-edits it to generate a coherent and fluent final text. We perform extensive empirical analysis and ablation studies on few- shot and zero-shot settings across 4 datasets. Automatic and human evaluation shows that the proposed hierarchical approach is consis- tently capable of achieving state-of-the-art re- sults when compared to previous work.1 1 Introduction The recent development of large pretrained lan- guage models (PLMs; i.e. BERT (Devlin et al., 2019), GPT-3 (Brown et al., 2020), T5 (Raffel et al., 2020)) has caused a shift of interest in the research community towards domain adaptation and transfer learning. For the task of concept-to- text natural language generation (NLG), wherein the aim is to generate a natural language text that describes the semantic content of an abstract struc- tured machine-readable input (Meaning Represen- tation; MR), transfer learning from PLMs has be- come a popular and high performing approach with 13 out of the 15 participating teams in the latest 1Code and scripts are available at https://github. com/huawei-noah/noah-research/NLP/ PLM-Lex PLM-Lex PLM-Lex PLM-Aggr PLM-Aggr PLM-Aggr PLM-Postedit AGGREGATION LEXICALISATION POSTEDIT Figure 1: Structure of Hierarchical Recurrent Aggrega- tive Generation (HRAG). The lexicalisation PLM gen- erates one sub-phrase per attribute-value pair. The ag- gregation PLM recurrently combines sub-phrases and the post-edit PLM rephrases them into a fluent output. WebNLG+ Shared Task (Ferreira et al., 2020) em- ploying a fine-tuned pretrained model as their main submitted system. Specifically, T5-based systems achieved a human evaluation ranking on par with the ground truth in terms of fluency and adequacy. Transfer learning from PLMs also enables training on few-shot and zero-shot settings, i.e. when suf- ficient in-domain data are unavailable. Prominent and relevant examples include machine translation (Zoph et al., 2016; Brown et al., 2020) and NLG for task-oriented dialogues (Peng et al., 2020). This paper focuses on concept-to-text NLG, where recent machine learning and in extension transfer learning approaches adopt an end-to-end 2167 OFFER ( stylist name = Atelier Salon Willow Glen ) OFFER ( city = San Jose ) INFORM ( …

Content truncated to 500 words