Delta vs. N-Gram Tracing: Evaluating the Robustness of Authorship Attribution Methods

Delta vs. N-Gram Tracing: Evaluating the Robustness of Authorship Attribution Methods Thomas Proisl∗, Stefan Evert∗, Fotis Jannidis†, Christof Sch¨och‡, Leonard Konle†, Steffen Pielstr¨om† ∗Friedrich-Alexander-Universit¨at Erlangen-N¨urnberg, †Julius-Maximilians-Universit¨at W¨urzburg, ‡Universit¨at Trier ∗Bismarckstr. 6, 91054 Erlangen, †Am Hubland, 97074 W¨urzburg, ‡Universit¨atsring 15, 54296 Trier ∗{thomas.proisl, stefan.evert}@fau.de, †{fotis.jannidis, christof.schoech, leonard.konle}@uni-wuerzburg.de, pielstroem@biozentrum.uni-wuerzburg.de, ‡schoech@uni-trier.de Abstract Delta measures are a well-established and popular family of authorship attribution methods, especially for literary texts. N-gram tracing is a novel method for authorship attribution designed for very short texts, which has its roots in forensic linguistics. We evaluate the performance of both methods in a series of experiments on English, French and German literary texts, in order to investigate the relationship between authorship attribution accuracy and text length as well as the composition of the comparison corpus. Our results show that, at least in our setting, both methods require relatively long texts and are furthermore highly sensitive to the choice of authors and texts in the comparison corpus. Keywords: authorship attribution, stylometry, evaluation 1. Introduction Authorship attribution, i. e. the identification of the true au- thor of a text of unknown or disputed authorship based on quantitatively measured linguistic evidence (Juola, 2006; Koppel et al., 2009; Stamatatos, 2009), has applications in many fields, including literary studies, history, forensic lin- guistics and corpus stylistics. It is based on the assumption that individual writers have idiosyncratic habits of language use (which they are usually not aware of) that lead to stylis- tic similarities between texts written by the same author. A wide range of stylometric features has been proposed to capture these idiosyncrasies, ranging from relative frequen- cies of function words to measures of vocabulary richness and syntactic complexity. Based on such feature vectors, a disputed text can then be attributed to the most similar of a set of candidate authors. If we want to apply authorship attribution methods in real- world settings, e. g. in forensic linguistics, the reliability and robustness of the methods are of utmost importance. Various factors can have an impact on the methods, raising questions such as: To what extent does authorship attribution accuracy depend on the length of the disputed text and the size of the corpus against which it is compared? Is there a mini- mum text length below which results become too unreliable? What impact does the composition of the comparison corpus have? Are the methods robust with respect to the selection of authors and texts for the comparison corpus? In this paper, we try to answer those questions, at least partially, for Delta and N-gram tracing, two particularly simple but very successful authorship attribution methods that only rely on word- and character-level features. 2. Background and Related Work 2.1. Delta Measures Delta measures (Burrows, 2002; Argamon, 2008) are a popu- lar family of authorship attribution methods. They represent texts as simple bags-of-words, focusing on the n most fre- quent words (nMFW) in the corpus. Word frequencies are standardized to z-scores across the corpus, with a mean of 0 and standard deviation of 1 …

Content truncated to 500 words