Neural Duplicate Question Detection without Labeled Training Data

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1607–1617, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1607 Neural Duplicate Question Detection without Labeled Training Data Andreas R¨uckl´e and Nafise Sadat Moosavi and Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP) Department of Computer Science, Technische Universit¨at Darmstadt www.ukp.tu-darmstadt.de Abstract Supervised training of neural models to dupli- cate question detection in community Ques- tion Answering (cQA) requires large amounts of labeled question pairs, which are costly to obtain. To minimize this cost, recent works thus often used alternative methods, e.g., ad- versarial domain adaptation. In this work, we propose two novel methods: (1) the auto- matic generation of duplicate questions, and (2) weak supervision using the title and body of a question. We show that both can achieve improved performances even though they do not require any labeled data. We provide com- prehensive comparisons of popular training strategies, which provides important insights on how to ‘best’ train models in different sce- narios. We show that our proposed approaches are more effective in many cases because they can utilize larger amounts of unlabeled data from cQA forums. Finally, we also show that our proposed approach for weak supervision with question title and body information is also an effective method to train cQA answer selec- tion models without direct answer supervision. 1 Introduction The automatic detection of question duplicates in community Question Answering (cQA) forums is an important task that can help users to more effec- tively find existing questions and answers (Nakov et al., 2017; Cao et al., 2012; Xue et al., 2008; Jeon et al., 2005), and to avoid posting similar questions multiple times. Neural approaches to duplicate de- tection typically require large quantities of labeled question pairs for supervised training—i.e., labeled pairs of duplicate questions that can be answered with the same information.1 1For example, “Nautilus shortcut for new blank files?” and “How do you create a new document keyboard shortcut?” are titles of labeled duplicate questions from AskUbuntu.com. In practice, it is often difficult to obtain such data because of the immense manual effort that is required for annotation. A large number of cQA forums thus do not contain enough labeled data for supervised training of neural models.2 Therefore, recent works have used alternative train- ing methods. This includes weak supervision with question-answer pairs (Qiu and Huang, 2015), semi-supervised training (Uva et al., 2018), and adversarial domain transfer (Shah et al., 2018). An important limitation of these methods is that they still rely on substantial amounts of labeled data— either thousands of duplicate questions (e.g., from a similar source domain in the case of domain trans- fer) or large numbers of question-answer pairs. Fur- thermore, unsupervised methods rely on encoder- decoder architectures that impose limitations on the model architectures and they often fall short of the performances that are achieved with super- vised training (Lei et al., 2016), or they need …

Content truncated to 500 words