MyFixit: An Annotated Dataset, Annotation Tool, and Baseline Methods for Information Extraction from Repair Manuals

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2120–2128 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2120 MyFixit: An Annotated Dataset, Annotation Tool, and Baseline Methods for Information Extraction from Repair Manuals Nima Nabizadeh1, Dorothea Kolossa1, Martin Heckmann2 1Cognitive Signal Processing Group, Ruhr University Bochum, Germany , 2Honda Research Institute Europe GmbH, Germany 1{nima.nabizadeh, dorothea.kolossa}@rub.de 2martin.heckmann@honda-ri.de Abstract Text instructions are among the most widely used media for learning and teaching. Hence, to create assistance systems that are capable of supporting humans autonomously in new tasks, it would be immensely productive, if machines were enabled to extract task knowledge from such text instructions. In this paper, we, therefore, focus on information extraction (IE) from the instructional text in repair manuals. This brings with it the multiple challenges of information extraction from the situated and technical language in relatively long and often complex instructions. To tackle these challenges, we introduce a semi-structured dataset of repair manuals. The dataset is annotated in a large category of devices, with information that we consider most valuable for an automated repair assistant, including the required tools and the disassembled parts at each step of the repair progress. We then propose methods that can serve as baselines for this IE task: an unsupervised method based on a bags-of-n-grams similarity for extracting the needed tools in each repair step, and a deep-learning-based sequence labeling model for extracting the identity of disassembled parts. These baseline methods are integrated into a semi-automatic web-based annotator application that is also available along with the dataset. Keywords: semi-structured dataset, information extraction, instructional text, procedural task, repair manual, semi-automatic annotation web tool 1. Introduction Since the emergence of the earliest writing systems, textual instructions have always been among the most ubiquitous means of transferring procedural knowledge. Nowadays, free instructions are available on the Web for numerous tasks of human life, be they easy every-day hacks, recipes, or professional-level instructions on using or repairing tech- nical devices of the highest complexity. For the latter, in- structions might be written in a complex and technical style, and they, in general, often use situated language (Malmaud et al., 2014). The instructional text is regularly divided into multiple steps, which should be performed in a specified se- quence. Consequently, there could be a sequence of objects that the person interacts with during the process. Extract- ing such practical information from text presents a unique challenge in the domain of information extraction (IE). The type and structure of the extracted information also de- pend on the characteristics of the task and the intended use of the information. In this work, we focus on IE from a highly complex type of instructions: repair manuals. More specifically, we focus on extracting the pieces of informa- tion that a collaborative repair assistant would benefit most from, in order to support a human in a repair task. A fruitful collaboration among agents depends on multiple prerequisites, one of them being sufficient shared …

Content truncated to 500 words