Short Text Authorship Attribution via Sequence Kernels, Markov Chains and Author Unmasking: An Investigation

Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pages 482–491, Sydney, July 2006. c⃝2006 Association for Computational Linguistics Short Text Authorship Attribution via Sequence Kernels, Markov Chains and Author Unmasking: An Investigation Conrad Sanderson and Simon Guenter Australian National University, Canberra, ACT 0200, Australia National ICT Australia, Locked Bag 8001, ACT 2601, Australia conrad.sanderson.@.anu.edu.au, simon.guenter.@.rsise.anu.edu.au Abstract We present an investigation of recently proposed character and word sequence kernels for the task of authorship attribu- tion based on relatively short texts. Per- formance is compared with two corre- sponding probabilistic approaches based on Markov chains. Several configurations of the sequence kernels are studied on a relatively large dataset (50 authors), where each author covered several topics. Utilis- ing Moffat smoothing, the two probabilis- tic approaches obtain similar performance, which in turn is comparable to that of char- acter sequence kernels and is better than that of word sequence kernels. The results further suggest that when using a realistic setup that takes into account the case of texts which are not written by any hypoth- esised authors, the amount of training ma- terial has more influence on discrimination performance than the amount of test mate- rial. Moreover, we show that the recently proposed author unmasking approach is less useful when dealing with short texts. 1 Introduction Applications of authorship attribution include pla- giarism detection (e.g. college essays), deducing the writer of inappropriate communications that were sent anonymously or under a pseudonym (e.g. threatening or harassing e-mails), as well as resolving historical questions of unclear or disputed authorship. Specific examples are the Federalist papers (Hanus and Hagenauer, 2005; Mosteller, 1984) and the forensic analysis of the Unabomber manifesto (Foster, 2001). Within the area of automatic author at- tribution, recently it has been shown that encouraging performance can be achieved via the use of probabilistic models based on n-grams (Clement and Sharp, 2003) and Markov chains of characters and words (Peng et al., 2004). Diederich et al. (2003) showed that Support Vector Machines (SVMs), using the bag-of-words kernel, can obtain promising performance, while in an- other study, SVMs with kernels based on character collocations obtained mixed performance (Cor- ney, 2003). Gamon (2004) utilised SVMs with syntactic and semantic features to obtain relatively minor accuracy improvements over the use of function word frequencies and part-of-speech trigrams. Koppel & Schler (2004) proposed a word-level heuristic, resembling recursive feature elimination used for cancer classification (Guyon et al., 2002; Huang and Kecman, 2005), to obtain author unmasking curves. The curves were processed to obtain feature vectors that were in turn classified in a traditional SVM setting. The studies listed above have several limita- tions. In (Clement and Sharp, 2003), a rudimen- tary probability smoothing technique was used to handle n-grams which were unseen during the training phase. In the dataset used by (Peng et al., 2004) each author tended to stick to one or two topics, raising the possibility that the discrimina- tion was based on topic rather than by author style. …

Content truncated to 500 words