Analysis of Tree-Structured Architectures for Code Generation

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4382–4391 August 1–6, 2021. ©2021 Association for Computational Linguistics 4382 Analysis of Tree-Structured Architectures for Code Generation Samip Dahal Adyasha Maharana Department of Computer ScienceMohit Bansal University of North Carolina at Chapel Hill {sdpmas, adyasha, mbansal}@cs.unc.edu Abstract Code generation is the task of generating code snippets from input user specifications in nat- ural language. Leveraging the linguistically- motivated hierarchical structure of the input can benefit code generation, especially since the specifications are complex sentences con- taining multiple variables and operations over various data structures. Moreover, recent advances in Transformer architectures have led to improved performance with tree-to-tree style generation for other seq2seq tasks e.g., machine translation. Hence, we present an empirical analysis of the significance of input parse trees for code generation. We run text- to-tree, linearized tree-to-tree, and structured tree-to-tree models, using constituency-based parse trees as input, where the target is Ab- stract Syntax Tree (AST) of the code. We eval- uate our models on the Python-based code gen- eration dataset CoNaLa and a semantic pars- ing dataset ATIS. We find that constituency trees encoded using a structure-aware model improve performance for both datasets. We also provide an analysis of those aspects of the input parse trees which are most impact- ful. For instance, we find that structure-aware encodings are better at modelling inputs with multiple variables and capturing long-range dependencies for code generation.1 1 Introduction Code generation is the task of converting input user specifications written in natural language (NL) to code snippets in a target programming language. It is a task-driven variant of semantic parsing, which translates natural language input to formal machine- executable representation. Recent works have uti- lized the Abstract Syntax Tree (AST) - which is the syntactic tree representation of target source code 1Code available at: https://github.com/ sdpmas/TreeCodeGen. - to generate better code snippets (Yin and Neu- big, 2017, 2018; Sun et al., 2020; Rabinovich et al., 2017). The use of ASTs has achieved strong results but there has been relatively less work on utilizing the parse trees of the NL input. Constituency or dependency trees representing the syntactic struc- ture of input can be leveraged to perform sub-tree alignment with corresponding AST of target code and benefit the downstream task. Hence, in this paper, we present several tree-to-tree models for the code generation task that convert the parse tree representation of NL input to AST representation of target source code. First, we base our model on the Transformer architecture (Vaswani et al., 2017). However, the standard Transformer is not designed to preserve the tree structure of the input parse trees. Hence, to better encode the trees, we modify a structure-aware Tree Transformer model (Nguyen et al., 2020) for the tree-to-tree code gen- eration task. We focus on constituency-based parse trees in this paper because of space constraints as this is a short paper. Moreover, as pointed out by Nguyen et al. (2020), there is little evidence of constituency structures being learned implicitly in …

Content truncated to 500 words