The AusNC Project: Plans, Progress and Implications for Language Technology

The AusNC Project: Plans, Progress and Implications for Language Technology Simon Musgrave Michael Haugh Linguistics Program School of Languages and Linguistics Monash University Griffith University VIC 3800 Australia QLD 4111 Australia Simon.Musgrave@arts.monash.edu.au m.haugh@griffith.edu.au Abstract In the last eighteen months, a consensus has emerged from researchers in various disci- plines that a vital piece of research infrastruc- ture is lacking in Australia, namely, a substantial collection of computerised lan- guage data. A result of this consensus is an ini- tiative aimed at the establishment of an Australian National Corpus. The progress of this initiative is presented in this paper, along with discussion of some important design is- sues and a consideration of how the initiative relates to the field of language technology in Australia. 1 Introduction Large-scale corpora are becoming an increasingly important resource in language research, including many sub-disciplines within language technology. An initiative has developed over the last year or more which aims to construct such a corpus as a key element of research infrastructure for Australia. A national repository of language data would have significant value as research infrastructure for a number of research communities in Australia and overseas, thereby increasing access to Australian language data and widening the global integration of research on language in Australia. It would fa- cilitate collaborative ventures in collecting new language data to support multimodal research in human communication and it would consolidate presently scattered and relatively inaccessible col- lections of historical language data where possible within the Australian National Corpus (AusNC). Such data is of interest not only to researchers in linguistics and applied linguistics, but also to mem- bers of the wider Humanities and Social Sciences and informatics research communities who have an interest in Australian society. Such a large anno- tated language dataset would also provide invalu- able training data for work in natural language processing, speech recognition, and the further de- velopment of semi-automated annotation. In this paper, we will give a short overview of the progress of the initiative to date. This will be followed by an introduction to some design ques- tions which have been the subject of discussion in the preliminary phase of the project, and a consid- eration of some implications of the project for the field of language technology. 2 History and Recent Progress At least two projects can be considered to have made substantial contributions to corpus-building in Australia before the present initiative was launched. From 1986, the Australian Corpus of English was compiled at Macquarie University.1 This corpus consists of 500 text samples of (minimally) 2,000 words each, giving a total size of approximately 1,000,000 words. This corpus has been integrated into the International Corpus of English project. The Australian National Database of Spoken Lan- guage was collected in the years between 1991 and 1995.2 This corpus consists of recordings of various types of spoken language plus associated tran- scripts. Although some of the material in this col- lection is taken from pairs of speakers collaborating on a map task, the …

Content truncated to 500 words