Explainable Natural Language to Bash Translation using Abstract Syntax Tree

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 258–267 November 10–11, 2021. ©2021 Association for Computational Linguistics 258 Explainable Natural Language to Bash Translation using Abstract Syntax Tree Shikhar Bharadwaj and Shirish Shevade Department of Computer Science and Automation, Indian Institute of Science, Bangalore, KA 560012, India {shikharb,shirish}@iisc.ac.in Abstract Natural language processing for program syn- thesis has been widely researched. In this work, we focus on generating Bash commands from natural language invocations with expla- nations. We propose a novel transformer based solution by utilizing Bash Abstract Syntax Trees and manual pages. Our method incor- porates tree structure information in the trans- former architecture and provides explanations for its predictions via alignment matrices be- tween user invocation and manual page text. Our method performs on par with the state of the art performance on Natural Language Con- text to Command task and performs better than fine-tuned T5 and Seq2Seq models. 1 Introduction Natural Language Processing (NLP) of program- ming language source code is a well-established field with a recent resurgence of using NLP tech- niques to assist programming. Advances have been made in code generation (Svyatkovskiy et al., 2020), code summarization (Ahmad et al., 2020), bug detection (Gupta et al., 2019), code transla- tion (Lachaux et al., 2020) etc. However, there is a lack of explainable methods for assisting software developers. Bash is a Unix command language with numer- ous utilities (like cut, tr, sed etc) for interacting with the Operating System. It is difficult for a novice developer to remember the purpose and syn- tax of these Bash utilities. Usually, to understand the intended usage of a utility, one looks up rele- vant manual page using man, apropos and info. However, these pages have been described as "fre- quently hard to read", byzantine and formal (Cozzie et al., 2011) and finding relevant information from them itself becomes a challenging task. Hence it gets difficult for a new user to learn Bash. Previ- ous works in this area (Agarwal et al., 2021, 2020; Gros, 2019) have explored Transformer (Vaswani et al., 2017), Sequence to Sequence with atten- tion (Bahdanau et al., 2015) (Seq2Seq), grammar guided methods and retrieval based methods for translating Natural Language to Bash commands. However, these methods are predictive and do not provide explanations or reasoning for their predic- tions. These methods propose a black-box solution to the translation problem relying on only parallel data. There is a lack of user trust in such predic- tions. Motivated by these challenges, we propose an explainable neural machine translation model. To the best of our knowledge, ours is the first method to explore the use of manual page data for assisting translation. Besides translating natural language invocations to Bash code, our model also provides alignment matrices between user invocations and manual text, which explain the predictions by the model. Such alignments could be helpful to gain developer confidence in using the system. At the same time, it can help improve the learning curve for Bash …

Content truncated to 500 words