Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6118–6129 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 6118 Evidence-Aware Inferential Text Generation with Vector Quantised Variational AutoEncoder Daya Guo1∗, Duyu Tang2, Nan Duan2, Jian Yin1, Daxin Jiang3 and Ming Zhou2 1 The School of Data and Computer Science, Sun Yat-sen University. Guangdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, P.R.China 2 Microsoft Research Asia, Beijing, China 3 Microsoft Search Technology Center Asia, Beijing, China {guody5@mail2,issjyin@mail}.sysu.edu.cn {dutang,nanduan,djiang,mingzhou}@microsoft.com Abstract Generating inferential texts about an event in different perspectives requires reasoning over different contexts that the event occurs. Existing works usually ignore the context that is not explicitly provided, resulting in a context-independent semantic representation that struggles to support the generation. To address this, we propose an approach that au- tomatically finds evidence for an event from a large text corpus, and leverages the evi- dence to guide the generation of inferential texts. Our approach works in an encoder- decoder manner and is equipped with a Vector Quantised-Variational Autoencoder, where the encoder outputs representations from a distri- bution over discrete variables. Such discrete representations enable automatically selecting relevant evidence, which not only facilitates evidence-aware generation, but also provides a natural way to uncover rationales behind the generation. Our approach provides state-of- the-art performance on both Event2Mind and ATOMIC datasets. More importantly, we find that with discrete representations, our model selectively uses evidence to generate different inferential texts. 1 Introduction Inferential text generation aims to understand daily- life events and generate texts about their underlying causes, effects, and mental states of event partici- pants, which is crucial for automated commonsense reasoning. Taking Figure 1 as an example, given an event “PersonX reads PersonY’s diary”, the cause of the participant “PersonX” is to “obtain Person Y’s secrets” and the mental state of “PersonX” is “guilty”. Standard approaches for inferential text generation (Rashkin et al., 2018; Sap et al., 2019; Bosselut et al., 2019; Du et al., 2019) typically only ∗ Work done while this author was an intern at Microsoft Research. PersonX stole PersonY’s diary secretly PersonY invites PersonX to read his diary know more about PersonY PersonX feels Event Background Inferences obtain PersonY’s secrets PersonX wants to PersonX reads PersonY’s diary guilty curious Figure 1: An examples of inferential text generation on mental states of event participants. We show two kinds of reasonable inferences for the event under different background knowledge that is absent in the dataset. take the event as the input, while ignoring the back- ground knowledge that provides crucial evidence to generate reasonable inferences. For example, if the background knowledge of this example is “Per- sonY invites PersonX to read his diary”, the outputs should be different. In this paper, we present an evidence-aware gen- erative model, which first retrieves relevant evi- dence from a large text corpus and then leverages retrieved evidence to guide the generation of infer- ential texts. Our model is built upon Transformer- based (Vaswani et al., 2017) …

Content truncated to 500 words