A Legal Perspective on Training Models for Natural Language Processing

A Legal Perspective on Training Models for Natural Language Processing Richard Eckart de Castilho♣, Giulia Dore♠, Thomas Margoni♠, Penny Labropoulou♥, Iryna Gurevych♣ ♣Technische Universit¨at Darmstadt, Department of Computer Science, UKP Lab ♠University of Glasgow, School of Law - CREATe Centre ♥Athena RC, Institute for Language and Speech Processing Abstract A significant concern in processing natural language data is the often unclear legal status of the input and output data/resources. In this paper, we investigate this problem by discussing a typical activity in Natural Language Processing: the training of a machine learning model from an annotated corpus. We examine which legal rules apply at relevant steps and how they affect the legal status of the results, especially in terms of copyright and copyright-related rights. Keywords: Copyright, Licensing, Machine Learning, Annotated Corpora 1. Introduction The state-of-the-art in many areas of Natural Language Pro- cessing (NLP) and Text Mining (TM) is based on Machine Learning (ML). Algorithms learn abstract probabilistic mod- els from texts annotated with labels (e.g. named entities, part-of-speech tags, sentiment tags, etc.) in order to predict such labels on unseen text. NLP tasks usually require the deployment of multiple components each using specialised models. As training models can be tedious and computation- ally intensive, pre-trained models are a valuable resource. However, the legal status of these models is often dubious, as in many cases it is unclear (a) whether a model can be trained from a corpus in absence of specific authorisation, (b) which licence (if any) can or must be assigned to them, and (c) if and in which cases the licence(s) of the original corpus and annotations affect the licensing of a model. This legal uncertainty often constitutes a hurdle, if not a real barrier for the development of research infrastructures and repositories, such as CLARIN1 or OpenMinTeD,2 where models are shared and used. In this paper, we explore the process of training a model both from an NLP and a legal perspective. We discuss under which circumstances annotated corpora may be used for training a model and if their legal status may restrict the choice of licences that can be applied to the trained model. We use EU copyright law as the main reference, although the analysis may find application beyond the EU (and may need some degree of adjustment in different EU Member States). 2. Background This section starts introducing the typical actions and re- sources involved in ML models construction and deploy- ment and successively discusses the relevant legal concepts. 2.1. NLP perspective Models are constructed through a training process involving a learning algorithm and training data to learn from. The model captures abstract probabilistic characteristics from 1https://www.clarin.eu 2https://www.openminted.eu the training data, which can then be used to predict the learned labels on unseen data. For illustration purposes, we focus on Named Entity Recognition (NER), an example of a sequence classification task. In general, constructing a model consists of the following steps: (1) corpus compilation, (2) corpus pre-processing, (3) corpus annotation, and (4) training of …

Content truncated to 500 words