Behavior Cloned Transformers are Neurosymbolic Reasoners

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2777–2788 May 2-6, 2023 ©2023 Association for Computational Linguistics Behavior Cloned Transformers are Neurosymbolic Reasoners Ruoyao Wang‡, Peter Jansen‡, Marc-Alexandre Côté♣, Prithviraj Ammanabrolu♢ ‡University of Arizona, Tucson, AZ ♣Microsoft Research Montréal ♢Allen Institute for AI, Seattle, WA {ruoyaowang,pajansen}@arizona.edu macote@microsoft.com, raja@allenai.org Abstract In this work, we explore techniques for aug- menting interactive AI agents with information from symbolic modules, much like humans use tools like calculators and GPS systems to as- sist with arithmetic and navigation. We test our agent’s abilities in text games—challenging benchmarks for evaluating the multi-step rea- soning abilities of game agents in grounded, language-based environments. Our experimen- tal study indicates that injecting the actions from these symbolic modules into the action space of a behavior cloned transformer agent increases performance on four text game bench- marks that test arithmetic, navigation, sorting, and common sense reasoning by an average of 22%, allowing an agent to reach the highest possible performance on unseen games. This action injection technique is easily extended to new agents, environments, and symbolic mod- ules.1 1 Introduction Interactive fiction games (or text games) evalu- ate AI agents abilities to perform complex multi- step reasoning tasks in interactive environments that are rendered exclusively using textual descrip- tions. Agents typically find these games challeng- ing due to the complexities of the tasks combined with the reasoning limitations of contemporary models. Overall performance is generally low, with agents currently solving only 30% of clas- sic interactive fiction games such as Zork (Am- manabrolu and Hausknecht, 2020; Yao et al., 2021; Atzeni et al., 2022). Similarly, reframing bench- marks such as question answering into text games where agents must interactively reason with their environment and make their reasoning steps ex- plicit causes performance to substantially decrease (Wang et al., 2022), highlighting both the capacity 1We release our system as open source, available at http: //github.com/cognitiveailab/neurosymbolic/ Task Description: Your task is to solve the math problem. Then, pick up the item with the same quantity as the math problem answer, and place it in the box. Symbolic Module Text Game Environment Agent Text Observation: You are in the kitchen. You see a math problem, 3 pears, 2 bananas, ... read math problem Action Symbolic Module Text Game Environment Agent Text Observation: Your task is to solve the following math problem: divide 22 by 11 div 22 11 Action Symbolic Module Text Game Environment Agent Text Observation: The result of dividing 22 by 11 is 2. take 2 bananas Action Step 1 Step 2 Step 3 Figure 1: An overview of our approach on an example game evaluating arithmetic ability. At each step, the agent receives an observation from the environment, then takes an action. By providing actions that interface to symbolic modules (such as a calculator), the agent is able to use external knowledge to help solve the task. of this methodology to evaluate multi-step reason- ing, and the limitations of current language models. …

Content truncated to 500 words