Analyzing the Perceived Severity of Cybersecurity Threats Reported on Social Media

Proceedings of NAACL-HLT 2019, pages 1380–1390 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1380 Analyzing the Perceived Severity of Cybersecurity Threats Reported on Social Media Shi Zong1, Alan Ritter1, Graham Mueller2 and Evan Wright3 1The Ohio State University, OH, USA 2Leidos Inc., VA, USA 3FireEye LLC, CA, USA {zong.56, ritter.1492}@osu.edu muellerwg@leidos.com evan.wright@fireeye.com Abstract Breaking cybersecurity events are shared across a range of websites, including secu- rity blogs (FireEye, Kaspersky, etc.), in addi- tion to social media platforms such as Face- book and Twitter. In this paper, we investi- gate methods to analyze the severity of cyber- security threats based on the language that is used to describe them online. A corpus of 6,000 tweets describing software vulnerabili- ties is annotated with authors’ opinions toward their severity. We show that our corpus sup- ports the development of automatic classifiers with high precision for this task. Furthermore, we demonstrate the value of analyzing users’ opinions about the severity of threats reported online as an early indicator of important soft- ware vulnerabilities. We present a simple, yet effective method for linking software vulner- abilities reported in tweets to Common Vul- nerabilities and Exposures (CVEs) in the Na- tional Vulnerability Database (NVD). Using our predicted severity scores, we show that it is possible to achieve a Precision@50 of 0.86 when forecasting high severity vulnerabilities, significantly outperforming a baseline that is based on tweet volume. Finally we show how reports of severe vulnerabilities online are pre- dictive of real-world exploits.1 1 Introduction Software vulnerabilities are flaws in computer sys- tems that leave users open to attack; vulnerabili- ties are generally unknown at the time a piece of software is first published, but are gradually iden- tified over time. As new vulnerabilities are discov- ered and verified they are assigned CVE numbers (unique identifiers), and entered into the National Vulnerability Database (NVD).2 To help prioritize 1Our code and data are available at https://github.com/viczong/ cybersecurity threat severity analysis. 2 https://nvd.nist.gov/ Figure 1: Example tweet discussing the dirty copy-on- write (COW) security vulnerability in the Linux kernel. response efforts, vulnerabilities in the NVD are as- signed severity scores using the Common Vulner- ability and Scoring System (CVSS). As the rate of discovered vulnerabilities has increased in recent years,3 the need for efficient identification and pri- oritization has become more crucial. However, it is well known that a large time delay exists be- tween the time a vulnerability is first publicly dis- closed to when it is published in the NVD; a re- cent study found that the median delay between the time a vulnerability is first reported online and the time it is published in the NVD is seven days; also, 75% of threats are first disclosed online giv- ing attackers time to exploit the vulnerability.4 In this paper we present the first study of whether natural language processing techniques can be used to analyze users’ opinions about the severity of software vulnerabilities reported on- line. We present a corpus of 6,000 …

Content truncated to 500 words