Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3498–3511 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3498 Regulatory Compliance through Doc2Doc Information Retrieval: A case study in EU/UK legislation where text similarity has limitations Ilias Chalkidis † ‡ Manos Fergadiotis † ‡ Nikolaos Manginas † Eva Katakalou ♭ ∗ Prodromos Malakasiotis † ‡ † EY AI Centre of Excellence in Document Intelligence, NCSR “Demokritos” ‡ Department of Informatics, Athens University of Economics and Business ♭ Department of International, European and Area Studies, Panteion University [Ilias.Chalkidis, Fergadiotis.Manos]@ey.com [Nikolaos.Manginas, Prodromos.Malakasiotis]@ey.com [ichalkidis, mfergadiotis]@iit.demokritos.gr [nmanginas, pmalakasiotis]@iit.demokritos.gr e.katakalou@panteion.gr Abstract Major scandals in corporate history have urged the need for regulatory compliance, where or- ganizations need to ensure that their controls (processes) comply with relevant laws, regula- tions, and policies. However, keeping track of the constantly changing legislation is difficult, thus organizations are increasingly adopting Regulatory Technology (RegTech) to facilitate the process. To this end, we introduce regula- tory information retrieval (REG-IR), an appli- cation of document-to-document information retrieval (DOC2DOC IR), where the query is an entire document making the task more chal- lenging than traditional IR where the queries are short. Furthermore, we compile and re- lease two datasets based on the relationships between EU directives and UK legislation. We experiment on these datasets using a typical two-step pipeline approach comprising a pre- fetcher and a neural re-ranker. Experimenting with various pre-fetchers from BM25 to k near- est neighbors over representations from sev- eral BERT models, we show that fine-tuning a BERT model on an in-domain classification task produces the best representations for IR. We also show that neural re-rankers under- perform due to contradicting supervision, i.e., similar query-document pairs with opposite la- bels. Thus, they are biased towards the pre- fetcher’s score. Interestingly, applying a date filter further improves the performance, show- casing the importance of the time dimension. ∗ The contribution of Ms. Eva Katakalou was restricted to the creation and the validation of the datasets as well as to the authoring of the corresponding parts of the manuscript. Figure 1: Number of legislative acts issued by the EU per year. The gold color of the bars indicates how many of the published acts are amendments to older ones. 1 Introduction Major scandals in corporate history, from Enron to Tyco International, Olympus, and Tesco,1 have led to the emergence of stricter regulatory mandates and highlighted the need for regulatory compliance where organizations need to ensure that they com- ply with relevant laws, regulations, and policies (Lin, 2016). However, keeping track of the con- stantly changing legislation (Figure 1) is hard, thus organizations are increasingly adopting Regulatory Technology (RegTech) to facilitate the process. Typically, a compliance regimen includes three distinct but related types of measures, corrective, detective, and preventive (Sadiq and Governatori, 1www.theguardian.com/business/2015/ju l/21/the-worlds-biggest-accounting-scan dals-toshiba-enron-olympus 3499 2015). Corrective measures are usually undertaken when new regulations are introduced to update ex- isting controls. Detective measures, ensure “after- the-fact” compliance, i.e., following a procedure, a …

Content truncated to 500 words