Generating Instructions at Different Levels of Abstraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 2802–2813 Barcelona, Spain (Online), December 8-13, 2020 2802 Generating Instructions at Different Levels of Abstraction Arne K¨ohn∗ and Julia Wichlacz∗ and ´Alvaro Torralba∗∗ and Daniel H¨oller∗ and J¨org Hoffmann∗ and Alexander Koller∗ ∗Saarland Informatics Campus Saarland University {koehn, koller}@coli.uni-saarland.de {wichlacz, hoeller, hoffmann}@cs.uni-saarland.de ∗∗Department of Computer Science Aalborg University alto@cs.aau.dk Abstract When generating technical instructions, it is often convenient to describe complex objects in the world at different levels of abstraction. A novice user might need an object explained piece by piece, while for an expert, talking about the complex object (e. g. a wall or railing) directly may be more succinct and efficient. We show how to generate building instructions at different levels of abstraction in Minecraft. We introduce the use of hierarchical planning to this end, a method from AI planning which can capture the structure of complex objects neatly. A crowdsourcing evaluation shows that the choice of abstraction level matters to users, and that an abstraction strategy which balances low-level and high-level object descriptions compares favorably to ones which don’t. 1 Introduction Technical instructions in complex environments can often be stated at different levels of abstraction. For instance, a natural language generation (NLG) system for tech support might instruct a human instruction follower (IF) to either plug “the broadband cable into the broadband filter” or “the thin white cable with grey ends into the small white box” (Janarthanam and Lemon, 2010). Depending on how much the IF knows about the domain, the first, high-level instruction may be difficult to understand, or the second, detailed instruction may be imprecise and annoyingly verbose. An effective instruction generation system will thus adapt the level of abstraction to the user. In this paper, we investigate the generation of instructions at different levels of abstraction in the context of the computer game “Minecraft”. Minecraft offers a virtual 3D environment in which the player can mine materials, craft items, and construct complex objects, such as buildings and machines. The user constructs these objects bit by bit from atomic blocks, and it is always possible to generate natural-language instructions which describe the placement of each individual block. However, it can be more effective to generate more high-level instructions. In the bridge-building example shown in Fig. 1, it is probably better to simply say “build a railing on the other side” instead of explaining where to place the seven individual blocks – provided the IF knows what a railing looks like. Minecraft is the best-selling video game of all time (200 million users), which means that there is a large pool of potential experimental subjects for evaluating NLG systems. Minecraft has been used previously as a platform for experimentation in AI and in particular for NLG (Narayan-Chen et al., 2019; K¨ohn and Koller, 2019). We present an instruction giving (IG) system which guides the user in constructing complex objects in Minecraft, such as houses and bridges. The system consists of two parts: a hierarchical planning system …

Content truncated to 500 words