Mise en Place: Unsupervised Interpretation of Instructional Recipes

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 982–992, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Mise en Place: Unsupervised Interpretation of Instructional Recipes Chlo´e Kiddon†, Ganesa Thandavam Ponnuraj‡, Luke Zettlemoyer†, and Yejin Choi† † Computer Science & Engineering, University of Washington, Seattle, WA {chloe, lsz, yejin}@cs.washington.edu ‡ Department of Computer Science, Stony Brook University, Stony Brook, NY gthandavam@gmail.com Abstract We present an unsupervised hard EM ap- proach to automatically mapping instruc- tional recipes to action graphs, which de- fine what actions should be performed on which objects and in what order. Recov- ering such structures can be challenging, due to unique properties of procedural lan- guage where, for example, verbal argu- ments are commonly elided when they can be inferred from context and disambigua- tion often requires world knowledge. Our probabilistic model incorporates aspects of procedural semantics and world knowl- edge, such as likely locations and selec- tional preferences for different actions. Experiments with cooking recipes demon- strate the ability to recover high quality action graphs, outperforming a strong se- quential baseline by 8 points in F1, while also discovering general-purpose knowl- edge about cooking. 1 Introduction Instructional language describes how to achieve a wide variety of goals, from traveling successfully to a desired location to cooking a particular dish for dinner. Despite the fact that such language is important to our everyday lives, there has been rel- atively little effort to design algorithms that can automatically convert it into an actionable form. Existing methods typically assume labeled train- ing data (Lau et al., 2009; Maeta et al., 2015) or access to a physical simulator that can be used to test understanding of the instructions (Branavan et al., 2009; Chen and Mooney, 2011; Bollini et al., 2013). In this paper, we present the first approach for unsupervised learning to interpret instructional recipes using text alone, with application to cook- ing recipes. Given a recipe, our task is to segment it into text spans that describe individual actions and con- struct an action graph whose nodes represent ac- tions and edges represent the flow of arguments across actions, for example as seen in Fig. 1. This task poses unique challenges for semantic anal- ysis. First, null arguments and ellipses are ex- tremely common (Zwicky, 1988). For example, sentences such as “Bake for 50 minutes” do not explicitly mention what to bake or where. Second, we must reason about how properties of the phys- ical objects are changed by the described actions, for example to correctly resolve what the phrase “the wet mixture” refers to in a baking recipe. Al- though linguistic context is important to resolving both of these challenges, more crucial is common sense knowledge about how the world works, in- cluding what types of things are typically baked or what ingredients could be referred to as “wet.”1 These challenges seemingly present a chicken and egg problem — if we had a high quality se- mantic analyzer for instructions we could …

Content truncated to 500 words