Corpus Based Classification of Text in Australian Contracts

Corpus Based Classification of Text in Australian Contracts Michael Curtotti Australian National University Canberra, Australia michael.curtotti@anu.edu.au Eric Mccreath Australian National University Canberra, Australia eric.mccreath@anu.edu.au Abstract Written contracts are a fundamental framework for commercial and coopera- tive transactions and relationships. Lim- ited research has been published on the ap- plication of machine learning and natural language processing (NLP) to contracts. In this paper we report the classification of components of contract texts using ma- chine learning and hand-coded methods. Authors studying a range of domains have found that combining machine learning and rule based approaches increases accu- racy of machine learning. We find similar results which suggest the utility of consid- ering leveraging hand coded classification rules for machine learning. We attained an average accuracy of 83.48% on a multi- class labelling task on 20 contracts com- bining machine learning and rule based approaches, increasing performance over machine learning alone. 1 Introduction Contracts govern economic transactions and com- mercial or organisational relationships from triv- ial purchases to major national infrastructure projects. Any large organisation (whether private or public) must unavoidably invest significant re- sources in developing and concluding contracts, as the contracts it enters define its legal relations with organisations and individuals with which it inter- acts. As one author has noted, contracts are an integral part of any business enterprise and it is “difficult to overstate their importance to the busi- ness world” (Khoury and Yamouni, 2007, p16). In addition, drafting contracts represents a major eco- nomic activity for the legal industry. Ambiguity is an unresolved problem in natu- ral language processing. It is also an important problem affecting the drafting and operation of contracts. The costs of ambiguity in a contract can be enormous. The presence of ambiguity in- creases the risk of litigation, project failure and loss of commercial relationships. Instances of lit- igated ambiguity are used in drafting manuals as instructional tales for the unwary drafter (Aitken and Butt, 2004, p37). Our future research goal is to investigate ap- proaches that will help identify such ambiguity, however, a precursor for addressing the problem of ambiguity is accurate labelling or classification of the constituent parts of contract documents.1 In this paper we report work on classifying ‘lines’ in contracts into 32 classes. We compare three dif- ferent classification approaches in order to deter- mine which approach may provide the best per- formance with respect to this task: supervised machine learning; a hand-crafted rule based tag- ger; or combining hand-crafted rules based tag- ging with machine learning. 2 Background Natural language processing involves applying a pipeline of transformations to text. A typical first step is to segment the text into its sentences, af- ter which further processing is applied (Jurafsky and Martin, 2009, p 69). This works well in usual English prose, but is not necessarily well suited to contracts because of the heavy use of layout to em- body structure and semantics. In a contract, a sen- tence may occur within a single paragraph of text, or may …

Content truncated to 500 words