UniEvent: Unified Generative Model with Multi-Dimensional Prefix for Zero-Shot Event-Relational Reasoning

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7088–7102 July 9-14, 2023 ©2023 Association for Computational Linguistics Unified Generative Model with Multi-Dimensional Prefix for Zero-Shot Event-Relational Reasoning Zhengwei Tao1 Zhi Jin1∗ Haiyan Zhao1 Chengfeng Dou1 Yongqiang Zhao1 Tao Shen2 Chongyang Tao3 1Peking University, 2FEIT, University of Technology Sydney, 3Microsoft {tttzw, yongqiangzhao}@stu.pku.edu.cn, {zhijin,zhhy.sei,chengfengdou}@pku.edu.cn tao.shen@uts.edu.au, chotao@microsoft.com Abstract Reasoning about events and their relations at- tracts surging research efforts since it is re- garded as an indispensable ability to fulfill various event-centric or common-sense reason- ing tasks. However, these tasks often suffer from limited data availability due to the labor- intensive nature of their annotations. Conse- quently, recent studies have explored knowl- edge transfer approaches within a multi-task learning framework to address this challenge. Although such methods have achieved accept- able results, such brute-force solutions struggle to effectively transfer event-relational knowl- edge due to the vast array of inter-event rela- tions (e.g. temporal, causal, conditional) and reasoning formulations (e.g. discriminative, ab- ductive, ending prediction). To enhance knowl- edge transfer and enable zero-shot generaliza- tion among various combinations, in this work we propose a novel unified framework, called UNIEVENT. Inspired by prefix-based multi- task learning, our approach organizes event re- lational reasoning tasks into a coordinate sys- tem with multiple axes, representing inter-event relations and reasoning formulations. We then train a unified text-to-text generative model that utilizes coordinate-assigning prefixes for each task. By leveraging our adapted prefixes, our unified model achieves state-of-the-art or com- petitive performance on both zero-shot and su- pervised reasoning tasks, as demonstrated in extensive experiments. 1 Introduction An ‘event’ is defined as a semantic molecule to explain the states or actions of a person, entity, or thing (Zhou et al., 2022). In natural language literature, it is usually represented as a span in ∗*Corresponding author. Temporal RE Question: What is the relation between documenting and voiced? Answer: BEFORE Document: Memorial is famous for documenting human rights abuses in Russia . the US embassy in Moscow has voiced concern and asked the Russian government for an explanation. RE QA TEMP CA Temporal QA Question: What happens before that the US embassy has voiced concern? Answer: Memorial is famous for documenting human rights abuses in Russia . Causal RE Question: What is the relation between documenting and voiced? Answer: CAUSE Causal QA Question: What causes that the US embassy has voiced concern? Answer: Memorial is famous for documenting human rights abuses in Russia . Existing Approaches UniEvent Figure 1: Illustration of knowledge transfer types across event-relational reasoning tasks. Existing approaches can only achieve inter-relation or inter-formulation trans- fer while UNIEVENT succeeds in all. narrative text (e.g., sentences, paragraphs or doc- uments), which is composed of an event trigger (e.g., predicate) and its arguments (e.g., subject, object, adverbial modifier). Based on the semantic unit at the event level, a broad spectrum of event- relational reasoning tasks was presented to learn various inter-event relations (e.g., temporal, causal, conditional) and thus enable commonsense or cog- nitive …

Content truncated to 500 words