Learning Bill Similarity with Annotated and Augmented Corpora of Bills

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10048–10064 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 10048 Learning Bill Similarity with Annotated and Augmented Corpora of Bills Jiseon Kim†, Elden Griggs§, In Song Kim§, Alice Oh† †School of Computing, KAIST §Department of Political Science, MIT jiseon_kim@kaist.ac.kr {emgriggs, insong}@mit.edu alice.oh@kaist.edu Abstract Bill writing is a critical element of represen- tative democracy. However, it is often over- looked that most legislative bills are derived, or even directly copied, from other bills. De- spite the significance of bill-to-bill linkages for understanding the legislative process, existing approaches fail to address semantic similari- ties across bills, let alone reordering or para- phrasing which are prevalent in legal docu- ment writing. In this paper, we overcome these limitations by proposing a 5-class classifica- tion task that closely reflects the nature of the bill generation process. In doing so, we con- struct a human-labeled dataset of 4,721 bill- to-bill relationships at the subsection-level and release this annotated dataset to the research community. To augment the dataset, we gen- erate synthetic data with varying degrees of similarity, mimicking the complex bill writ- ing process. We use BERT variants and apply multi-stage training, sequentially fine-tuning our models with synthetic and human-labeled datasets. We find that the predictive perfor- mance significantly improves when training with both human-labeled and synthetic data. Finally, we apply our trained model to infer section- and bill-level similarities. Our analy- sis shows that the proposed methodology suc- cessfully captures the similarities across legal documents at various levels of aggregation. 1 1 Introduction In the U.S. and other democracies, a bill becomes a law through a complex political process, which includes a series of political debates and compro- mises within their legislative branch. A consistent pattern that arises from this political interaction is the emergence of several lengthy omnibus bills which cover a range of topics/special interest and combine parts of numerous smaller bills. In fact, 1We release our human-annotated & synthetic dataset and code at https://github.com/hikoseon12/ learning-bill-similarity most bills tend to be a mixture of a number of other bills, whether enacted or “dead.” Furthermore, in the U.S., many bills are modified and re-introduced between the two chambers of Congress. Uncov- ering these complex relationships among bills is important because it helps scholars better under- stand the legislative process, one of the most im- portant parts of representative democracy. In this paper, we develop a methodology to characterize this relationship across bills. Specifically, we lever- age semantic similarities between subsections to infer bill linkages using annotated and augmented synthetic corpora. Previous studies have investigated this is- sue in both political science (Wilkerson et al., 2015; Linder et al., 2020) and computer sci- ence (Burgess et al., 2016a,b). To find similar bills, they use either a simple bag-of-words model or the Smith-Waterman local alignment algorithm (SWAlign) (Gotoh, 1982) . Although this latter ap- proach is useful in identifying exact text matches, it cannot capture semantic similarities across bills that arise from …

Content truncated to 500 words