Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

Findings of the Association for Computational Linguistics: ACL 2022, pages 97 - 111 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Controlled Text Generation Using Dictionary Prior in Variational Autoencoders Xianghong Fang1, Jian Li2∗, Lifeng Shang2, Xin Jiang2, Qun Liu2, Dit-Yan Yeung1 1The Hong Kong University of Science and Technology 2Huawei Noah’s Ark Lab xfangam@connect.ust.hk, dyyeung@cse.ust.hk {lijian703, shang.lifeng, jiang.xin, qun.liu}@huawei.com Abstract While variational autoencoders (VAEs) have been widely applied in text generation tasks, they are troubled by two challenges: insuffi- cient representation capacity and poor control- lability. The former results from the posterior collapse and restrictive assumption, which im- pede better representation learning. The lat- ter arises as continuous latent variables in tra- ditional formulations hinder VAEs from inter- pretability and controllability. In this paper, we propose Dictionary Prior (DPrior), a new data-driven prior that enjoys the merits of ex- pressivity and controllability. To facilitate con- trolled text generation with DPrior, we pro- pose to employ contrastive learning to separate the latent space into several parts. Extensive experiments on both language modeling and controlled text generation demonstrate the ef- fectiveness of the proposed approach. 1 Introduction As one of the representative deep generative mod- els, variational autoencoders (VAEs) (Kingma and Welling, 2014) have been widely applied in text generation tasks, such as dialog generation (Wu et al., 2020; Zhao et al., 2017), machine transla- tion (Shah and Barber, 2018; McCarthy et al., 2020; Sheng et al., 2020) and poetry generation (Li et al., 2018b; Yi et al., 2020). Despite the success, VAEs still suffer from two challenges: insufficient repre- sentation capacity and poor controllability. The challenge of insufficient representation ca- pacity in variational models arises from two aspects. One is the posterior collapse, a notorious issue that generally exists in VAEs especially serious in auto- regressive text generation (Bowman et al., 2016), which leads to degenerate local optimums during the training of VAEs (He et al., 2019). Another is the restrictive assumption for priors and variational ∗* Corresponding author. This work was done when Xi- anghong was an intern at Huawei Noah’s Ark Lab. Attributes Samples Positive this is followed by good movies, great food. Negative for me it looks crappy and understaffed. Present this restaurant has an excellent view. Past i was able to get the delicious sushi! Table 1: Examples of controlled text generation in sec- ond column where sentence attributes indicated by col- ored words are consistent with user-specified attributes in the first column. posteriors (Ding and Gimpel, 2021), which gen- erally follow Gaussian distribution and spherical Gaussian distributions with diagonal co-variance matrices, respectively (Higgins et al., 2017; He et al., 2019; Li et al., 2019a). Such predefined forms would hinder VAEs from larger optimization space (Fang et al., 2019), thus restricting the ex- pressivity of the model (Ding and Gimpel, 2021) and further leading to the posterior collapse (Fang et al., 2019). Therefore, a potential solution is to try more expressive distribution forms for priors and variational posteriors to improve the representation capacity (Fang et al., …

Content truncated to 500 words