Few-shot Subgoal Planning with Language Models

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5493 - 5506 July 10-15, 2022 ©2022 Association for Computational Linguistics Few-shot Subgoal Planning with Language Models Lajanugen Logeswaran∗, Yao Fu‡, Moontae Lee∗†, Honglak Lee∗‡ LG AI Research∗, University of Illinois at Chicago†, University of Michigan‡ Abstract Pre-trained language models have shown suc- cessful progress in many text understanding benchmarks. This work explores the capability of these models to predict actionable plans in real-world environments. Given a text instruc- tion, we show that language priors encoded in pre-trained models allow us to infer fine- grained subgoal sequences. In contrast to re- cent methods which make strong assumptions about subgoal supervision, our experiments show that language models can infer detailed subgoal sequences from few training sequences without any fine-tuning. We further propose a simple strategy to re-rank language model predictions based on interaction and feedback from the environment. Combined with pre- trained navigation and visual reasoning com- ponents, our approach demonstrates compet- itive performance on subgoal prediction and task completion in the ALFRED benchmark compared to prior methods that assume more subgoal supervision. 1 Introduction Developing autonomous agents that can complete specific tasks given goal descriptions embodies human-level intelligence. Successful agents in this setting require multiple reasoning capabilities in- cluding natural language understanding, visual rea- soning, and acting over long temporal horizons. Training black-box models that map instructions and observations to suitable actions has proven to be difficult due to challenges in interpreting and reasoning with multimodal information, especially in the absence of strong supervision. Thus, general- ization in this setting demands effective strategies for planning, exploration, and incorporating feed- back from the environment. Generalization in human agents, on the other hand, stems from our ability to naturally brainstorm ∗Correspondence to llajan@lgresearch.ai Move a heated slice of apple to the white table with shelving below 1) slice apple, heat in microwave, put in side table. 2) slice apple, heat in microwave, put in shelf. 3) slice apple, heat in microwave, put in dining table. … 1) slice apple, heat in microwave, put in dining table. 2) slice apple, heat in microwave, put in side table. 3) slice apple, heat in microwave, put in shelf. … In-context learning with language model Put a rinsed slice of apple on the table = slice apple, clean it, put in … Put a plate with a watch on it on the table = pick up watch, put in plate, .. … Re-rank based on interaction with environment Candidate hypotheses Re-ranked hypotheses … turn left pickup knife put shelf success/failure feedback Query Instruction Figure 1: Overview of subgoal prediction approach. (a) A pre- trained language model prompted with a sequence of training instances, i.e., (instruction, subgoal sequence) pairs, and a query instruction predicts top-k hypotheses using beam search. (b) These predictions are then re-ranked by incorporating information about the environment. abstract subgoals, better calibrating executable ac- tions and their sequences. Planning at the level of subgoals …

Content truncated to 500 words