SemEval-2018 Task 8: Semantic Extraction from CybersecUrity REports using Natural Language Processing (SecureNLP)

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 697–706 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics SemEval-2018 Task 8: Semantic Extraction from CybersecUrity REports using Natural Language Processing (SecureNLP) Peter Phandi Amila Silva Wei Lu 8 Somapah Road, Singapore 487372 Singapore University of Technology and Design {peter phandi,amila silva,wei lu}@sutd.edu.sg Abstract This paper describes the SemEval 2018 shared task on semantic extraction from cybersecu- rity reports, which is introduced for the first time as a shared task on SemEval. This task comprises four SubTasks done incrementally to predict the characteristics of a specific mal- ware using cybersecurity reports. To the best of our knowledge, we introduce the world’s largest publicly available dataset of annotated malware reports in this task. This task received in total 18 submissions from 9 participating teams. 1 Introduction As a result of the world getting more connected and digitized, cyber attacks become increasingly common and pose serious issues for the society. More recently in 2017, a ransomware called Wan- naCry, which has the capability to lock down the data files using strong encryption, spread around the world targeting public utilities and large cor- porations (Mohurle and Patil, 2017). Another ex- ample is the botnet known as Mirai, which used infected Internet of Things (IoT) devices to dis- able Internet access for millions of users in the US West Coast (US-CERT, 2016) through large- scale Distributed Denial of Service (DDoS) at- tacks. The impact levels of these attacks is ranging from simple ransomware on personal laptops (An- dronio et al., 2015) to taking over the control of moving cars (Checkoway et al., 2011). Along with the importance of cybersecurity in today’s context, there is an increasing potential for substantial contribution in cybersecurity using nat- ural language processing (NLP) techniques, even though this has not been significantly addressed. We introduced this task as a shared task on Se- mEval for the first time with the intention of mo- tivating NLP researchers for this critical research Figure 1: Annotated sentence and sentence frag- ment from MalwareTextDB. Such annotations provide semantic-level information to the text. area. Even though there exists a large repository of malware related texts online, the sheer volume and diversity of these texts make it difficult for NLP researchers to quickly move to this research field. Another challenge is that most of the data is unannotated. Lim et al. (2017) has introduced a dataset of annotated malware reports for facil- itating future NLP work in cybersecurity. In the light of that, we improved Lim’s malware dataset to create, to the best of our knowledge, the world’s largest publicly available dataset of annotated mal- ware reports. The aim of our annotation is to mark the words and phrases in malware reports that de- scribe the behaviour and capabilities of the mal- ware and assign them to some certain categories. Most of the machine learning efforts in the task of malware detection were based on the sys- tem calls. Rieck et al. …

Content truncated to 500 words