Interchange Formats for Visualization: LIF and MMIF

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7230–7237 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7230 Interchange Formats for Visualization: LIF and MMIF Kyeongmin Rim∗, Kelley Lynch∗, Marc Verhagen∗, Nancy Ide†, James Pustejovsky∗ ∗ Brandeis University, Department of Computer Science {krim,kmlynch}@brandeis.edu, marc@cs.brandeis.edu, jamesp@brandeis.edu † Vassar College, Department of Computer Science ide@cs.vassar.edu Abstract Promoting interoperrable computational linguistics (CL) and natural language processing (NLP) application platforms and interchange- able data formats have contributed improving discoverabilty and accessbility of the openly available NLP software. In this paper, we discuss the enhanced data visualization capabilities that are also enabled by inter-operating NLP pipelines and interchange formats. For adding openly available visualization tools and graphical annotation tools to the Language Applications Grid (LAPPS Grid) and Computational Linguistics Applications for Multimedia Services (CLAMS) toolboxes, we have developed interchange formats that can carry annotations and metadata for text and audiovisual source data. We descibe those data formats and present case studies where we successfully adopt open-source visualization tools and combine them with CL tools. Keywords: Interchange formats, Interoperability, Data visualization, Multi-media annotation 1. Introduction In this paper, we discuss the enhanced data visualization ca- pabilities enabled by the open-source natural language pro- cessing (NLP) tools (either separately or pipelined in appli- cation workflows) as well as openly available visualization toolkits. While popular text-based search and navigation applications such as ElasticSearch+Kibana stack (ELK)1 or Solr2 are beginning to provide some Human Language Technology (HLT)-enabled indexing that can then be vi- sualized (e.g., named entity recognition (NER)), we ar- gue that a more robust and generic capability for visual- izing data is enabled by adopting common rich and flexi- ble interchange formats from which the visualization can be mapped. To this end, we describe the use of the LAPPS Interchange Format (LIF) and the Multi-Media Interchange Format (MMIF) as pivots from which visualizations can be driven. Specifically, we discuss two different scenarios for the vi- sualization of data: (i) visualization of an individual docu- ment and its annotations; and (ii) visualization of a collec- tion of documents that is processed offline in order to ul- timately provide a display of collection-wide phenomena. We demonstrate how visualization of an individual docu- ment and its annotations is performed by converting be- tween pivoting interchange formats (specifically LIF and/or MMIF) and application specific formats. Using interchange formats enables the chaining of tools into a pipeline which generates automatic annotations that can then be visual- ized or modified by a human-in-the-loop in multiple an- notation environments. Additionally, we describe the func- tionality enabled by using LIF and MMIF as interchange formats within the Language Applications Grid (LAPPS Grid)-Galaxy and Computational Linguistics Applications for Multimedia Services (CLAMS) respectively. The sec- ond scenario involves the visualization of data characteris- tics that inhere over an entire collection. We illustrate how MMIF and LIF can facilitate visualization across a collec- 1https://www.elastic.co/ 2https://lucene.apache.org/solr/ tion of documents by being exported for indexing in docu- ment indexing tools …

Content truncated to 500 words