SMS Spam Detection Through Skip-gram Embeddings and Shallow Networks

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4193–4201 August 1–6, 2021. ©2021 Association for Computational Linguistics 4193 SMS Spam Detection Through Skip-gram Embeddings and Shallow Networks Gustavo Jos´e de Sousa, Daniel Carlos Guimar˜aes Pedronette, Jo˜ao Paulo Papa, Ivan Rizzo Guilherme S˜ao Paulo State University - UNESP, Brazil {gustavo.sousa,daniel.pedronette,joao.papa,ivan.guilherme}@unesp.br Abstract The drastic decrease in mobile SMS costs turned phone users more prone to spam mes- sages, usually with unwanted marketing or questionable content. As such, researchers have proposed different methods for detecting SMS spam messages. This paper presents a technique for embedding SMS messages into vector spaces that is suitable for spam detec- tion. The proposed approach relies on min- ing patterns that are relevant for distinguishing spam from legitimate messages. A subset of those patterns is used to construct a function that maps text messages into a multidimen- sional vector space. The extracted patterns are represented as skip-grams of token attributes, where a skip-gram can be seen as a generaliza- tion of the n-gram model that allows a distance greater than one between matched tokens in the text. We evaluate the proposed approach using the generated vectors for spam classi- fication on the UCI Spam Collection dataset. The experiments showed that our method com- bined with shallow networks reached accu- racy that is competitive with state-of-the-art approaches. 1 Introduction SMS spam detection is a very relevant task for mo- bile phone users. It can mitigate the annoyance caused by the invasive marketing applied through this platform and provide a more secure user expe- rience by blocking potentially harmful messages. In that sense, researchers have proposed different methodologies to detect SMS spam. Abdulhamid et al. (2017) and Abayomi-Alli et al. (2019), for instance, provide comprehensive reviews of the relevant datasets and techniques found in the litera- ture. Most recently, some researchers have success- fully applied deep learning techniques for SMS spam detection: Annareddy and Tammina (2019) provide a comparative study on using Convolu- tional Neural Networks (CNN) and Recurrent Neu- ral Networks (RNN); Roy et al. (2020) and Popovac et al. (2018) propose to CNN as well; Roy et al. (2020), Jain et al. (2018) and Raj et al. (2018) pro- pose using LSTM-based architectures; Ghourabi et al. (2020) use a hybrid CNN-LSTM model; Barushka and Hajek (2018) use regularized deep multi-layer perceptrons combined with a feature selection algorithm; and Wei and Nguyen (2020) propose the use of Lightweight Gated Recurrent Units (LGRU). Other works employ traditional classifiers to such a task (Fernandes et al., 2015; Fattahi and Mejri, 2021; Xia and Chen, 2020), including di- verse models like Support Vector Machines (SVM), Hidden Markov Models (HMM), Optimum-Path Forest (OPF), k-Nearest Neighbors (KNN), deci- sion trees, and ensembling approaches. Gupta et al. (2018) provide a comparative study using CNN and traditional machine learning architectures. SMS spam messages have a very characteristic textual style; many inform the recipient that she has won a prize or offer apparent great deals. As such, many words (e.g., “prize”, “won”, “free”, “bonus”, etc.) …

Content truncated to 500 words