Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue Response Generation Models by Causal Discovery

On the Role of Negative Precedent in Legal Outcome Prediction Josef Valvoda Ryan Cotterell Simone Teufel University of Cambridge, UK ETH Z¨urich, Switzerland {jv406,sht25}@cam.ac.uk ryan.cotterell@inf.ethz.ch Abstract Every legal case sets a precedent by develop- ing the law in one of the following two ways. It either expands its scope, in which case it sets positive precedent, or it narrows it, in which case it sets negative precedent. Legal outcome prediction, the prediction of positive outcome, is an increasingly popular task in AI. In con- trast, we turn our focus to negative outcomes here, and introduce a new task of negative outcome prediction. We discover an asym- metry in existing models’ ability to predict positive and negative outcomes. Where the state-of-the-art outcome prediction model we used predicts positive outcomes at 75.06 F1, it predicts negative outcomes at only 10.09 F1, worse than a random baseline. To address this performance gap, we develop two new models inspired by the dynamics of a court process. Our first model significantly improves posi- tive outcome prediction score to 77.15 F1 and our second model more than doubles the nega- tive outcome prediction performance to 24.01 F1. Despite this improvement, shifting focus to negative outcomes reveals that there is still much room for improvement for outcome pre- diction models. https://github.com/valvoda /Negative-Precedent-in -Legal-Outcome-Prediction 1 Introduction The legal system is inherently adversarial. Ev- ery case pitches two parties against each other: the claimant, who alleges their rights have been breached, and the defendant, who denies breach- ing those rights. For each claim of the claimant, their lawyer will produce an argument, for which the defendant’s lawyer will produce a counter- argument. In precedential legal systems (Black, 2019), the decisions in the past judgements are binding on the judges in deciding new cases.1 Therefore, both sides of the dispute will rely on the outcomes of previous cases to support their position (Duxbury, 2008; Lamond, 2016; Black, 2019). The claimant will assert that her circum- stances are alike those of previous claimants whose rights have been breached. The defendant, on the other hand, will allege that the circum- stances are in fact more alike those of unsuccess- ful claimants. The judge decides who is right, and by doing so establishes a new precedent. If it is the claimant who is successful in a particular claim, the precedent expands the law by including the new facts in its scope. If it is the defendant who is successful, the law is contracted by rejection of the new facts from its scope. The expansion or contraction is encoded in the case outcome; we will refer to them as positive outcome and negative outcome, respectively. Positive and negative outcomes are equally binding, which means that the same reasons that motivate the research of the positive outcome also apply to the negative outcome. Both are important for computational legal analysis, a fact that has been known at least since Lawlor (1963).2 How- ever, the de facto interpretation of precedent in today’s legal NLP landscape focuses …

Content truncated to 500 words