Stacking Classifiers for Anti-Spam Filtering of E-Mail

In Proceedings of the 6th Conference on Empirical Methods in Natural Language Processing (EMNLP 2001), L. Lee and D. Harman (Eds.), pp. 44–50, Carnegie Mellon University, Pittsburgh, PA, USA, 2001. Stacking classifiers for anti-spam filtering of e-mail Georgios Sakkis♦ ♦♦ ♦, Ion Androutsopoulos♣♣♣♣, Georgios Paliouras♣♣♣♣, Vangelis Karkaletsis♣♣♣♣, Constantine D. Spyropoulos♣♣♣♣, and Panagiotis Stamatopoulos♦ ♦ ♦♦ ♦Department of Informatics University of Athens TYPA Buildings, Panepistimiopolis GR-157 71 Athens, Greece e-mail: {stud0926, T.Stamatopoulos}@di.uoa.gr ♣Software and Knowledge Engineering Laboratory Institute of Informatics and Telecommunications National Centre for Scientific Research “Demokritos” GR-153 10 Ag. Paraskevi, Athens, Greece e-mail: {ionandr, paliourg, vangelis, costass}@iit.demokritos.gr Abstract We evaluate empirically a scheme for combining classifiers, known as stacked generalization, in the context of anti-spam filtering, a novel cost-sensitive application of text categorization. Unsolicited commercial e- mail, or “spam”, floods mailboxes, causing frustration, wasting bandwidth, and exposing minors to unsuitable content. Using a public corpus, we show that stacking can improve the efficiency of automatically induced anti-spam filters, and that such filters can be used in real- life applications. Introduction This paper presents an empirical evaluation of stacked generalization, a scheme for combining automatically induced classifiers, in the context of anti-spam filtering, a novel cost-sensitive application of text categorization. The increasing popularity and low cost of e- mail have intrigued direct marketers to flood the mailboxes of thousands of users with unsolicited messages, advertising anything, from vacations to get-rich schemes. These messages, known as spam or more formally Unsolicited Commercial E-mail, are extremely annoying, as they clutter mailboxes, prolong dial-up connections, and often expose minors to unsuitable content (Cranor & Lamacchia, 1998). Legal and simplistic technical counter- measures, like blacklists and keyword-based filters, have had a very limited effect so far.1 The success of machine learning techniques in text categorization (Sebastiani, 2001) has recently led to alternative, learning-based approaches (Sahami, et al. 1998; Pantel & Lin, 1998; Drucker, et al. 1999). A classifier capable of distinguishing between spam and non-spam, hereafter legitimate, messages is induced from a manually categorized learning collection of messages, and is then used to identify incoming spam e-mail. Initial results have been promising, and experiments are becoming more systematic, by exploiting recently introduced benchmark corpora, and cost-sensitive evaluation measures (Gomez Hidalgo, et al. 2000; Androutsopoulos, et al. 2000a, b, c). Stacked generalization (Wolpert, 1992), or stacking, is an approach for constructing classifier ensembles. A classifier ensemble, or committee, is a set of classifiers whose individual decisions are combined in some way to classify new instances (Dietterich, 1997). a Stacking combines multiple classifiers to induce higher-level classifier with improved performance. The latter can be thought of as the president of a committee with the ground-level classifiers as members. Each unseen incoming message is first given to the members; the president then decides on the category of the 1 Consult www.cauce.org, spam.abuse.net, and www.junkemail.org. message by considering the opinions of the members and the message itself. Ground-level classifiers often make different classification errors. Hence, a president that has successfully learned when to trust each of the members …

Content truncated to 500 words