MQDD: Pre-training of Multimodal Question Duplicity Detection for Software Engineering Domain

Proceedings of Recent Advances in Natural Language Processing, pages 824–835 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_089 824 MQDD: Pre-training of Multimodal Question Duplicity Detection for Software Engineering Domain Jan Pašek, Jakub Sido, Miloslav Konopík, Ondˇrej Pražák {pasekj,sidoj,konopik,ondfa}@kiv.zcu.cz NTIS – New Technologies for the Information Society, Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia, Plzeˇn, Czech Republic Abstract This work proposes a new pipeline for lever- aging data collected on the Stack Overflow website for pre-training a multimodal model for searching duplicates on question answer- ing websites. Our multimodal model is trained on question descriptions and source codes in multiple programming languages. We design two new learning objectives to improve dupli- cate detection capabilities. The result of this work is a mature, fine-tuned Multimodal Ques- tion Duplicity Detection (MQDD) model, ready to be integrated into a Stack Overflow search system, where it can help users find answers for already answered questions. Alongside the MQDD model, we release two datasets related to the software engineering domain. The first Stack Overflow Dataset (SOD) represents a massive corpus of paired questions and answers. The second Stack Overflow Duplicity Dataset (SODD) contains data for training duplicate detection models. 1 Introduction The benefits of Question-Answer (QA) networks for software developers such as the Stack Overflow website are widely exploited by professionals and beginners alike during the software creation pro- cess. Many solutions to various problems, short tu- torials, and other helpful tips can be found on these networks. However, access to this valuable source of information highly depends on users’ ability to search for the answers. In our paper, we intro- duce a multimodal method for detecting duplicate questions. Apart from the primary use to prevent posting duplicate questions, this technique can be directly used for better search. When users are posting already answered questions, they can get the answer immediately without the necessity to wait until someone else links the duplicate post or answers their question. The duplicate question detection task aims to classify whether two questions share the same in- tent. In other words, if two questions are duplicates, they relate to the same answer. The duplicate de- tection task is quite challenging since the classifier needs to distinguish tiny semantic nuances that can significantly change the desired answer. The posts in the QA networks for software de- velopment often intermix natural language with source code snippets. The great success of neural networks for Natural Language Processing (NLP) encourages us to build a bi-modal natural language (NL) and programming language (PL) encoder for duplicate detection (Wang et al., 2020) on question- answering platforms such as Stack Overflow. Current state-of-the-art NLP methods build on large pre-trained models, leveraging Trans- former architecture (Vaswani et al., 2017). The Transformer-based models such as BERT (Devlin et al., 2018), GPT (Brown et al., 2020), RoBERTa (Liu et al., 2019), or T5 (Raffel et al., 2019) are usually pre-trained on massive unlabeled corpora and applied to a task with much less training data afterward. We …

Content truncated to 500 words