Estimating the Strength of Authorship Evidence with a Deep-Learning-Based Approach

Abstract This study is the first likelihood ratio (LR)- based forensic text comparison study in which each text is mapped onto an embedding vector using RoBERTa as the pre-trained model. The scores obtained with Cosine distance and probabilistic linear discriminant analysis (PLDA) were calibrated to LRs with logistic regression; the quality of the LRs was assessed by log LR cost (𝐶𝑙𝑙𝑟). Although the documents in the experiments were very short (maximum 100 words), the systems reached the 𝐶𝑙𝑙𝑟values of 0.55595 and 0.71591 for the Cosine and PLDA systems, respectively. The effectiveness of deep-learning-based text representation is discussed by comparing the results of the current study to those of the previous studies of systems based on conventional feature engineering tested with longer documents. 1 Introduction In forensic science, the likelihood ratio (LR) framework has long been considered the logically and legally correct approach to interpreting the analysis of forensic evidence (Aitken and Stoney, 1991; Aitken and Taroni, 2004; Morrison, 2022; Robertson et al., 2016). The LR framework is standard in DNA typing. The community of forensic text comparison (FTC), commonly known as forensic authorship verification, recently recognised the importance of this framework (Grant, 2022). Despite the importance of the LR framework in forensic science, LR-based studies on textual evidence are still conspicuously rare (Ishihara, 2017, 2021; Ishihara and Carne, 2022). Many studies claim to be forensic but treat the problem as a usual authorship verification problem. However, there are important differences between conventional and forensic authorship verification. Conventional authorship verification aims to answer a verification problem. Forensic authorship verification aims to assist the fact finder in concluding the case, not answering the problem. Legally, giving an answer to a verification problem (even in a probabilistic term) equates to referring to the ultimate question of ‘guilty vs. not guilty’, which is only permitted for the fact finder. Logically, forensic scientists without all evidential information of the case cannot estimate the probability of a hypothesis from incomplete evidence. Thus, they cannot logically refer to the ultimate question. However, forensic scientists can logically and legally estimate the strength of evidence via LR (Aitken and Stoney, 1991; Aitken and Taroni, 2004; Robertson et al., 2016). LR is given in Equation (1). LR is the ratio of two conditional probabilities; one is the probability of evidence (𝐸) given the prosecution hypothesis (𝐻𝑙���) and the other is the probability of the same evidence given the defence hypothesis (𝐻𝑐���). 𝐿𝑅 = 𝑃(𝐸|𝐻𝑙���) 𝑃(𝐸|𝐻𝑐���) (1) The relative strength of the given evidence with respect to the competing hypotheses is reflected in the magnitude of the LR. The greater the LR value is than 1, the stronger support the evidence is considered to provide for the prosecution; the smaller the LR value is than 1, mutatis mutandis, for the defence hypothesis. It is very important to note that the LR is not a binary expression of truth. With an LR estimated as the strength of evidence, the fact finder’s belief regarding the hypotheses (quantified as prior odds) is raised to …

Content truncated to 500 words