Constructing Flow Graphs from Procedural Cybersecurity Texts

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3945–3957 August 1–6, 2021. ©2021 Association for Computational Linguistics 3945 Constructing Flow Graphs from Procedural Cybersecurity Texts Kuntal Kumar Pal∗, Kazuaki Kashihara∗, Pratyay Banerjee∗, Swaroop Mishra, Ruoyu Wang, Chitta Baral School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, {kkpal, kkashiha, pbanerj6, srmishr1, fishw, chitta}@asu.edu Abstract Following procedural texts written in natural languages is challenging. We must read the whole text to identify the relevant informa- tion or identify the instruction-flow to com- plete a task, which is prone to failures. If such texts are structured, we can readily visu- alize instruction-flows, reason or infer a partic- ular step, or even build automated systems to help novice agents achieve a goal. However, this structure recovery task is a challenge be- cause of such texts’ diverse nature. This paper proposes to identify relevant information from such texts and generate information flows be- tween sentences. We built a large annotated procedural text dataset (CTFW) in the cyberse- curity domain (3154 documents). This dataset contains valuable instructions regarding soft- ware vulnerability analysis experiences. We performed extensive experiments on CTFW with our LM-GNN model variants in multi- ple settings. To show the generalizability of both this task and our method, we also exper- imented with procedural texts from two other domains (Maintenance Manual and Cooking), which are substantially different from cyberse- curity. Our experiments show that Graph Con- volution Network with BERT sentence embed- dings outperforms BERT in all three domains. 1 Introduction Many texts in the real-world contain valuable in- structions. These instructions define individual steps of a process and help users achieve a goal (and corresponding sub-goals). Documents includ- ing such instructions are called procedural texts, ranging from simple cooking recipes to complex instruction manuals. Additionally, discussion in a shared forum or social media platform, teaching books, medical notes, sets of advice about social behavior, directions for use, do-it-yourself notices, ∗ These authors contributed equally to this work. itinerary guides can all be considered as procedu- ral texts (Delpech and Saint-Dizier, 2008). Most of these texts are in the form of natural languages and thus, lacking structures. We define structure as sentence-level dependencies that lead to a goal. These dependencies can vary based on the text- domain. Some examples of such dependencies are action traces, effects of an action, information leading to the action, and instruction order. Con- structing structured flow graphs out of procedural texts is the foundation for natural language under- standing and summarization, question-answering (QA) beyond factoid QA, automated workflow vi- sualization, and the recovery of causal relationships between two statements. By flow-graph we mean both information and action flows in a text. How- ever, the lack of structures in such texts makes them challenging to follow, visualize, extract inferences, or track states of an object or a sub-task, which ultimately makes constructing their flow graphs an insurmountable task. Procedural texts are common in cybersecurity, where security analysts document how to discover, exploit, and mitigate security vulnerabilities …

Content truncated to 500 words