An Effect of Background Population Sample Size on the Performance of a Likelihood Ratio-based Forensic Text Comparison System: A Monte Carlo Simulation with Gaussian Mixture Model

An Effect of Background Population Sample Size on the Performance of a Likelihood Ratio-based Forensic Text Comparison System: A Monte Carlo Simulation with Gaussian Mixture Model Shunichi Ishihara Department of Linguistics, The Australian National University shunichi.ishihara@anu.edu.au Abstract This is a Monte Carlo simulation-based study that explores the effect of the sample size of the background database on a likelihood ratio (LR)-based forensic text comparison (FTC) system built on multivariate authorship attrib- ution features. The text messages written by 240 authors who were randomly selected from an archive of chatlog messages were used in this study. The strength of evidence (= LR) was estimated using the multivariate kernel density likelihood ratio (MVKD) formula with a logistic-regression calibration. The results are reported along two points: the system per- formance (= accuracy) and the stability of per- formance based on the standard metric for LR-based systems; namely the log-likelihood- ratio cost (Cllr). It was found in this study that the system performance and its stability im- prove as a function of the sample size (= au- thor count) in the background database in a non-linear manner, and that the more features used for modelling, the more background data the system generally requires for optimal re- sults. The implications of the findings to the real casework are also discussed. 1 Introduction 1.1 Forensic text comparison and the likeli- hood-ratio framework The conceptual framework of likelihood ratio (LR) has received or has started receiving wide support from various areas of forensic comparative scienc- es as the logically and legally correct framework for assessing forensic evidence, and presenting the strength of the evidence (Balding, 2005; Evett et al., 1998; Marquis et al., 2011; Morrison, 2009; Neumann et al., 2007). Although forensic text comparison (FTC) is lagging behind other areas of forensic comparative sciences, studies in which the LR framework was applied to authorship attribu- tion have started emerging (Ishihara, 2012, 2014b). As expressed in equation (1), the LR, the quanti- fied strength of evidence, is a ratio of two condi- tional probabilities: one is the probability (p) of observed evidence (E) assuming that the prosecu- tion hypothesis is true (Hp) and the other is the probability of the same observed evidence assum- ing that the defence hypothesis (Hd) is true (Robertson & Vignaux, 1995). For FTC, for instance, it will be the probability of observing the difference (referred to as the evi- dence, E) between the offender’s and the suspect’s text messages if they had been produced by the same author (Hp) relative to the probability of ob- serving the same evidence (E) if they had come from different authors (Hd). In practice, an LR is estimated as a ratio of two terms: similarity and typicality, which correspond to the numerator and denominator of equation (1). Similarity means the similarity (or difference) be- tween the offender and the suspect samples (e.g. text messages). Typicality means, in general terms, the typicality (or atypicality) of the offender sam- ple against the relevant population. If the offender and the …

Content truncated to 500 words