The CLAMS Platform at Work: Processing Audiovisual Data from the American Archive of Public Broadcasting

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2498–2506 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2498 The CLAMS Platform at Work: Processing Audiovisual Data from the American Archive of Public Broadcasting Marc Verhagen, Kelley Lynch, Kyeongmin Rim, James Pustejovsky Department of Computer Science, Brandeis University, Waltham, Massachusetts, {verhagen,kmlynch,krim,jamesp}@brandeis.edu Abstract The Computational Linguistics Applications for Multimedia Services (CLAMS) platform provides access to computational content analysis tools for multimedia material. The version we present here is a robust update of an initial prototype implemen- tation from 2019. The platform now sports a variety of image, video, audio and text processing tools that interact via a common multi-modal representation language named MMIF (Multi-Media Interchange Format). We describe the overall architecture, the MMIF format, some of the tools included in the platform, the process to set up and run a workflow, visualizations included in CLAMS, and evaluate aspects of the platform on data from the American Archive of Public Broadcasting, showing how CLAMS can add metadata to mass-digitized multimedia collections, metadata that are typically only available implicitly in now largely unsearchable digitized media in archives and libraries. Keywords: Multimedia platforms, Web services, Archives, Interoperability 1. Introduction The CLAMS project provides access to tools that an- alyze multimedia content. The intended audiences for the platform are (1) researchers focused at processing multimedia content and (2) organizations like archives and libraries that seek to enrich their materials with metadata. It would be beneficial for those organiza- tions to be able to process their sources and improve the search functionality that they offer their customers and the public at large. The processing tools that allow that need to be affordable and relatively easy to use, and ideally it should be possible to adapt the tools to the data in question. In this paper we describe the first stable version of the CLAMS platform, following up on work in (Rim et al., 2019). We first give an overview of the system as it is now, pointing out the differences with its previous in- cantation. Then we describe some of the results of our work on the American Archive of Public Broadcasting (AAPB). We focus on what workflows were valuable for the AAPB data and which issues came up while processing the data. Finally we give an overview of related work. 2. CLAMS Architecture The main ingredients of the CLAMS platform are the applications, the REST API, the MMIF specifications and its associated Software Software Development Kit (SDK), the workflow managers, and the visualization components. An overview of those components and some of their interactions is shown in Figure 1. Figure 1: CLAMS Architecture 2.1. Applications Applications are processing services that are typically, but not always, based on off-the-shelf publicly avail- able processing tools with permissive licenses. We use the term application or app to refer to the software that takes a processing tool and embeds it as a CLAMS web service and we use tool to …

Content truncated to 500 words