Exploring Representation-level Augmentation for Code Search

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4924 - 4936 December 7-11, 2022 ©2022 Association for Computational Linguistics Exploring Representation-Level Augmentation for Code Search Haochen Li1 Chunyan Miao1,2∗ Cyril Leung1,2 Yanxian Huang3 Yuan Huang3 Hongyu Zhang4 Yanlin Wang3 1School of Computer Science and Engineering, Nanyang Technological University, Singapore 2China-Singapore International Joint Research Institute (CSIJRI), China 3School of Software Engineering, Sun Yat-sen University, China 4The University of Newcastle, Australia {haochen003,ascymiao,cleung}@ntu.edu.sg, huangyx353@mail2.sysu.edu.cn {huangyuan5,wangylin36}@mail.sysu.edu.cn, hongyu.zhang@newcastle.edu.au Abstract Code search, which aims at retrieving the most relevant code fragment for a given natural lan- guage query, is a common activity in software development practice. Recently, contrastive learning is widely used in code search research, where many data augmentation approaches for source code (e.g., semantic-preserving pro- gram transformation) are proposed to learn better representations. However, these aug- mentations are at the raw-data level, which requires additional code analysis in the pre- processing stage and additional training costs in the training stage. In this paper, we ex- plore augmentation methods that augment data (both code and query) at representation level which does not require additional data pro- cessing and training, and based on this we propose a general format of representation- level augmentation that unifies existing meth- ods. Then, we propose three new augmenta- tion methods (linear extrapolation, binary inter- polation, and Gaussian scaling) based on the general format. Furthermore, we theoretically analyze the advantages of the proposed aug- mentation methods over traditional contrastive learning methods on code search. We experi- mentally evaluate the proposed representation- level augmentation methods with state-of-the- art code search models on a large-scale pub- lic dataset consisting of six programming lan- guages. The experimental results show that our approach can consistently boost the perfor- mance of the studied code search models. Our source code is available at https://github. com/Alex-HaochenLi/RACS. 1 Introduction In software development, developers often search and reuse commonly used functionalities to im- prove their productivity (Nie et al., 2016; Shuai et al., 2020). With the growing size of large-scale codebases such as GitHub, retrieving semantically ∗Corresponding author. relevant code fragments accurately becomes in- creasingly important in this field (Allamanis et al., 2018; Liu et al., 2021). Traditional approaches (Nie et al., 2016; Yang and Huang, 2017; Rosario, 2000; Hill et al., 2011; Satter and Sakib, 2016; Lv et al., 2015; Van Nguyen et al., 2017) leverage information retrieval tech- niques to treat code snippets as natural language text and match certain terms in code with queries, hence suffering from the vocabulary mismatch problem (McMillan et al., 2011; Robertson et al., 1995). Deep siamese neural networks first embed queries and code fragments into a joint embedding space, then measure similarity by calculating dot product or cosine distance (Lv et al., 2015; Cam- bronero et al., 2019; Gu et al., 2021). Recently, with the popularity of large scale pre-training tech- niques, some big models for source code (Guo et al., 2021; Feng et al., 2020; Guo et al., 2022; Wang et al., 2021; Jain et al., …

Content truncated to 500 words