Hierarchical Entity Typing via Multi-level Learning to Rank

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 807–812 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 807 Unsupervised FAQ Retrieval with Question Generation and BERT Yosi Mass, Boaz Carmeli, Haggai Roitman and David Konopnicki IBM Research AI Haifa University, Mount Carmel, Haifa, HA 31905, Israel {yosimass,boazc,haggai,davidko}@il.ibm.com Abstract We focus on the task of Frequently Asked Questions (FAQ) retrieval. A given user query can be matched against the questions and/or the answers in the FAQ. We present a fully un- supervised method that exploits the FAQ pairs to train two BERT models. The two models match user queries to FAQ answers and ques- tions, respectively. We alleviate the missing labeled data of the latter by automatically gen- erating high-quality question paraphrases. We show that our model is on par and even outper- forms supervised models on existing datasets. 1 Introduction Many websites and online communities publish FAQ to help their users find relevant answers to common questions. An FAQ consists of pairs of questions and answers {(q, a)}. The FAQ retrieval task involves ranking {(q, a)} pairs for a given user query Q.1 Searching over FAQ can leverage mul- tifield indexing and retrieval (Karan and ˇSnajder, 2016). Hence, a user query Q may be matched with either the question field q, the answer field a or the concatenated field q+a (Karan and ˇSnajder, 2016). The association of questions to answers in the FAQ pairs, can be utilized as weak supervision, for training neural models to predict the similar- ity between user queries and answers (i.e., Q-to-a matching) (Gupta and Carvalho, 2019; Karan and ˇSnajder, 2018; Sakata et al., 2019). However, FAQ pairs by themselves do not provide the required labeled data for training a model to predict the as- sociation between user queries and FAQ questions (i.e., Q-to-q matching). Thus, a labeled dataset with user queries Q and their matching {(q, a)} 1Throughout this paper we use the term “question” (q) to denote a question within a given FAQ pair, and “query” (Q) to denote an issued user query. pairs is required for supervised learning (Gupta and Carvalho, 2019; Karan and ˇSnajder, 2018; Sakata et al., 2019). Such a dataset is usually manually generated or obtained from query-log mining. Yet, the construction of such a dataset either requires domain expertise (e.g., enriching the dataset with manually generated question paraphrases (Karan and ˇSnajder, 2018)) or assumes the availability of query-logs (Kim and Seo, 2006, 2008). Whenever such a dataset is unavailable, one must resort to utilizing unsupervised retrieval models for Q-to-q matching. Previous unsupervised FAQ retrieval models (Burke et al., 1997; Brill et al., 2002; Karan et al., 2013; Karan and ˇSnajder, 2018; Wu et al., 2005) have utilized so far “traditional” information retrieval techniques, such as lexical and semantic text matching, query expansion, etc. In this paper we overcome the aforementioned unsupervised gap, by using distant supervision to train neural models. Our method is composed of a combination of three unsupervised methods. Each …

Content truncated to 500 words