UMBC at SemEval-2018 Task 8: Understanding Text about Malware

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 878–884 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics UMBC at SemEval-2018 Task 8: Understanding Text about Malware Ankur Padia1, Arpita Roy1, Taneeya Satyapanich1, Francis Ferraro1, Shimei Pan1, Youngja Park2, Anupam Joshi1, Tim Finin1 1 University of Maryland, Baltimore County Baltimore, MD, 21250 USA {pankur1,arpita2,taneeya1,ferraro,shimei,joshi,finin}@umbc.edu 2 IBM T. J. Watson Research Center Yorktown Heights, New York, USA young park@us.ibm.com Abstract We describe the systems developed by the UMBC team for 2018 SemEval Task 8, Se- cureNLP (Semantic Extraction from Cyberse- cUrity REports using Natural Language Pro- cessing). We participated in three of the sub- tasks: (1) classifying sentences as being rele- vant or irrelevant to malware, (2) predicting to- ken labels for sentences, and (4) predicting at- tribute labels from the Malware Attribute Enu- meration and Characterization vocabulary for defining malware characteristics. We achieved F1 scores of 50.34/18.0 (dev/test), 22.23 (test- data), and 31.98 (test-data) for Task1, Task2 and Task2 respectively. We also make our cy- bersecurity embeddings publicly available at https://bit.ly/cybr2vec. 1 Introduction Task 8 for SemEval 2018 asked participants to work on a set of related sub-tasks involving ana- lyzing information from text about malware drawn from the Advanced Persistent Threats Notes col- lection (Blanda and Westcott, 2018) using the se- mantic framework found in the Malware Attribute Enumeration and Characterization language (Kir- illov et al., 2011; Beck et al., 2014). The task was composed of four related sub-tasks that could be part of a processing pipeline for an informa- tion extraction system for cybersecurity related text (Phandi et al., 2018). Subtask 1 required classifying a sentence as be- ing relevant or irrelevant for inferring malware ac- tions and capabilities. Subtask 2 involved predict- ing token labels for entities, actions and modifiers in sentences. Subtask 3, which we did not under- take, expanded on subtask 2 by asking participants to label relevant relations between the entities. Subtask 4 required predicting more detailed at- tribute labels, including ActionName, Capability, StrategicObjectives and TacticalObjectives, drawn from the MAEC vocabulary. One of our aims is to better understand the dif- ferences between cybersecurity text and general, non-cybersecurity text; another is to also better understand differences and variation within cyber- security texts. To that end, we focus on learn- ing and extracting better representations of the in- put reports. Specifically, for our approaches, we focus on approaches that incorporate additional, domain-specific knowledge into our system, and we use these enhanced representations and fea- tures in well-studied classification, representation, and sequence prediction models. 2 Subtask 1 In this section we describe our approaches to classify a given sentence as relevant or irrele- vant to malware. We used logistic regression (LR), multi-layer perceptron (MLP), and Long- Short Term Memory (Hochreiter and Schmidhu- ber, 1997, LSTM) as classifiers, and we used mul- tiple encoding schemes to represent features for the classification task. 2.1 Models We experimented with and evaluated three dif- ferent techniques for implementing the Subtask 1 relevance classifier. Each approach …

Content truncated to 500 words