A Comparative Study of Likelihood Ratio Based Forensic Text Comparison in Procedures: Multivariate Kernel Density vs. Gaussian Mixture Model-Universal Background Model

A Comparative Study of Two Procedures for Calculating Likelihood Ratio in Forensic Text Comparison: Multivariate Kernel Density vs. Gaussian Mixture Model-Universal Background Model Shunichi Ishihara Department of Linguistics Australian National University shunichi.ishihara@anu.edu.au Abstract We compared the performances of two proce- dures for calculating the likelihood ratio (LR) on the same set of text data. The first proce- dure was a multivariate kernel density (MVKD) procedure which has been success- fully applied to various types of forensic evi- dence, including glass fragments, handwriting, fingerprint, voice, and texts. The second pro- cedure was a Gaussian mixture model – uni- versal background model (GMM-UBM), which has been commonly used in forensic voice comparison (FVC) with so-called auto- matic features. Previous studies have applied the MVKD system to electronically-generated texts to estimate LRs, but so far no previous studies seem to have applied the GMM-UBM system to such texts. It has been reported that the latter GMM-UBM system outperforms the MVKD system in FVC. The data used for this study was chatlog messages collected from 115 authors, which were divided into test, background and development databases. Three different sample sizes of 500, 1500 and 2500 words were used to investigate how the per- formance is susceptible to the sample size. Re- sults show that regardless of sample size, the performance of the GMM-UBM system was better than that of the MVKD system with re- spect to both validity (= accuracy) (of which the metric is the log-likelihood-ratio cost, Cllr) and reliability (= precision) (of which the met- ric is the 95% credible interval, CI). 1 Introduction There are a large number of authorship analysis studies claiming to be forensic, particularly in the fields of computational linguistics and natural language processing (Iqbal et al. 2008, Iqbal et al. 2010, Lambers & Veenman 2009, Teng et al. 2004). Although they describe highly sophisti- cated statistical and computational methodolo- gies, many of them consider the problem as a classification problem: for example, whether a system correctly identifies text as having been written by the same author or by different au- thors, etc. However, it is critical to appreciate that the role of the forensic scientist in this situa- tion is not to give a definitive answer to the ques- tion of authorship or to give an opinion on the likely authorship (whether the incriminating text was written by the suspect or not). This is the task of the trier-of-fact. (Aitken 1995, Aitken & Stoney 1991, Aitken & Taroni 2004, Robertson & Vignaux 1995). The above point is empha- sised in the following quote. It is very tempting when assessing evidence to try to determine a value for the probabil- ity of guilt of a suspect, or the value for the odds in favour of guilt and perhaps even reach a decision regarding the suspect’s guilt. However, this is the role of the jury and/or judge. It is not the role of forensic scientist or statistical expert witness to give an opinion on this (Aitken 1995: 4). So, …

Content truncated to 500 words