Crossing Variational Autoencoders for Answer Retrieval

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5635–5641 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5635 Crossing Variational Autoencoders for Answer Retrieval Wenhao Yu†, Lingfei Wu‡, Qingkai Zeng†, Shu Tao‡, Yu Deng‡, Meng Jiang† †University of Notre Dame, Notre Dame, IN, USA ‡IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA †{wyu1, qzeng, mjiang2}@nd.edu ‡{wuli, shutao, dengy}@us.ibm.com Abstract Answer retrieval is to find the most aligned an- swer from a large set of candidates given a question. Learning vector representations of questions/answers is the key factor. Question- answer alignment and question/answer seman- tics are two important signals for learning the representations. Existing methods learned se- mantic representations with dual encoders or dual variational auto-encoders. The semantic information was learned from language mod- els or question-to-question (answer-to-answer) generative processes. However, the alignment and semantics were too separate to capture the aligned semantics between question and an- swer. In this work, we propose to cross vari- ational auto-encoders by generating questions with aligned answers and generating answers with aligned questions. Experiments show that our method outperforms the state-of-the- art answer retrieval method on SQuAD. 1 Introduction Answer retrieval is to find the most aligned an- swer from a large set of candidates given a ques- tion (Ahmad et al., 2019; Abbasiyantaeb and Momtazi, 2020). It has been paid increasing at- tention by the NLP and information retrieval com- munity (Yoon et al., 2019; Chang et al., 2020). Sentence-level answer retrieval approaches rely on learning vector representations (i.e., embeddings) of questions and answers from pairs of question- answer texts. The question-answer alignment and question/answer semantics are expected to be pre- served in the representations. In other words, the question/answer embeddings must reflect their se- mantics in the texts of being aligned as pairs. One popular scheme “Dual-Encoders” (also known as “Siamese network” (Triantafillou et al., 2017; Das et al., 2016)) has two separate encoders to generate question and answer embeddings and Table 1: The answer at the bottom of this table was aligned to 17 different questions at the sentence level. Question (1): What three stadiums did the NFL de- cide between for the game? Question (2): What three cities did the NFL consider for the game of Super Bowl 50? ... Question (17): How many sites did the NFL narrow down Super Bowl 50’s location to? Answer: The league eventually narrowed the bids to three sites: New Orleans Mercedes-Benz Superdome, Miami Sun Life Stadium, and the San Francisco Bay Area’s Levi’s Stadium. a predictor to match two embedding vectors (Cer et al., 2018; Yang et al., 2019). Unfortunately, it has been shown difficult to train deep encoders with the weak signal of matching prediction (Bow- man et al., 2015). Then there has been growing in- terests in developing deep generative models such as variational auto-encoders (VAEs) and genera- tive adversial networks (GANs) for learning text embeddings (Xu et al., 2017; Xie and Ma, 2019). As shown in Figure 1(b), the scheme of …

Content truncated to 500 words