Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4280–4295 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4280 Programming in Natural Language with fuSE: Synthesizing Methods from Spoken Utterances Using Deep Natural Language Understanding Sebastian Weigelt, Vanessa Steurer, Tobias Hey, and Walter F. Tichy Karlsruhe Institute of Technology Institute for Program Structures and Data Organization Karlsruhe, Germany weigelt@kit.edu, vanessa.steurer@web.de, hey@kit.edu, tichy@kit.edu Abstract The key to effortless end-user programming is natural language. We examine how to teach in- telligent systems new functions, expressed in natural language. As a first step, we collected 3168 samples of teaching efforts in plain En- glish. Then we built fuSE, a novel system that translates English function descriptions into code. Our approach is three-tiered and each task is evaluated separately. We first classify whether an intent to teach new functionality is present in the utterance (accuracy: 97.7% using BERT). Then we analyze the linguis- tic structure and construct a semantic model (accuracy: 97.6% using a BiLSTM). Finally, we synthesize the signature of the method, map the intermediate steps (instructions in the method body) to API calls and inject con- trol structures (F1: 67.0% with information retrieval and knowledge-based methods). In an end-to-end evaluation on an unseen dataset fuSE synthesized 84.6% of the method signa- tures and 79.2% of the API calls correctly. 1 Introduction Intelligent systems became rather smart lately. One easily arranges appointments by talking to a vir- tual assistant or controls a smart home through a conversational interface. Instructing a humanoid robot in this way no longer seems to be futuristic. For the time being, users can only access built-in functionality. However, they will soon expect to add new functionality themselves. For humans, the most natural way to communicate is by natural lan- guage. Thus, future intelligent systems must be programmable in everyday language. Today’s systems that claim to offer program- ming in natural language enable laypersons to is- sue single commands or construct short scripts (e.g. Mihalcea et al. (2006); Rabinovich et al. (2017)); usually no new functionality is learned. Only a few addressed learning new functionality from nat- ural language instructions (e.g. Le et al. (2013); Markievicz et al. (2017)). However, even recent approaches still either restrict the language or are (over-)fitted to a certain domain or application. We propose to apply deep natural language un- derstanding to the task of synthesizing methods from spoken utterances. Our approach combines modern machine learning techniques with infor- mation retrieval and knowledge-based methods to grasp the user’s intent. As a first step, we have per- formed a user study to investigate how laypersons teach new functionality with nothing but natural language. In a second step, we develop fuSE (Func- tion Synthesis Executor). fuSE translates teaching efforts into code. On the basis of the gathered data we constructed a three-tiered approach. We first determine, whether an utterance comprises an ex- plicitly stated intent to teach a new skill. Then, we decompose these teaching efforts into distinct semantic parts. We …

Content truncated to 500 words