The Language Resource Life Cycle: Towards a Generic Model for Creating, Maintaining, Using and Distributing Language Resources

The Language Resource Life Cycle: Towards a Generic Model for Creating, Maintaining, Using and Distributing Language Resources Georg Rehm DFKI GmbH Berlin, Germany georg.rehm@dfki.de Abstract Language Resources (LRs) are an essential ingredient of current approaches in Linguistics, Computational Linguistics, Language Technology and related fields. LRs are collections of spoken or written language data, typically annotated with linguistic analysis information. Different types of LRs exist, for example, corpora, ontologies, lexicons, collections of spoken language data (audio), or collections that also include video (multimedia, multimodal). Often, LRs are distributed with specific tools, documentation, manuals or research publications. The different phases that involve creating and distributing an LR can be conceptualised as a life cycle. While the idea of handling the LR production and maintenance process in terms of a life cycle has been brought up quite some time ago, a best practice model or common approach can still be considered a research gap. This article wants to help fill this gap by proposing an initial version of a generic Language Resource Life Cycle that can be used to inform, direct, control and evaluate LR research and development activities (including description, management, production, validation and evaluation workflows). Keywords: LR infrastructures and architectures, Metadata, Standards for LRs, Life cycle 1. Introduction Language Resources are an essential ingredient of current research and development approaches in Linguistics, Com- putational Linguistics, Language Technology and related fields. The respective methodologies usually involve the ap- plication of statistical machine learning techniques to vast amounts of data in order to train engines, generate models, create tools or to address novel, data-driven research ques- tions, among others. Generally speaking, Language Resources are collections of spoken or written language data (including audio and video), typically, but not necessarily, annotated with linguistic anal- ysis information. There are different types of Language Re- sources such as corpora, ontologies, lexicons, collections of spoken language data (audio), or collections that also in- clude video (multimedia, multimodal). Often, Language Resources are distributed with specific tools, documenta- tion, manuals or research publications. In order to ensure a certain level of sustainability and in- teroperability, standards and best practice approaches are used for many resources – either as is or including modi- fications – for the annotation of various linguistic analysis levels or metadata concerning the formal description of the whole resource. There are many cases in which resources have become popular and proven valuable for the research community. In these cases a resource often serves as the basis of additional research that explores related or maybe even completely different areas by extending the resource with additional language data or with additional linguistic analysis levels that have not been in scope with regard to the research question the resource was originally created for. The different phases that involve, among others, creating and distributing a Language Resource can be conceptu- alised as a life cycle. While the idea of handling the Lan- guage Resource production and maintenance process in terms of a life cycle has been brought up quite some …

Content truncated to 500 words