Multilingual Image Corpus: Annotation Protocol

Proceedings of Recent Advances in Natural Language Processing, pages 701–707 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_080 701 Multilingual Image Corpus: Annotation Protocol Svetla Koeva Institute for Bulgarian Language ”Prof. L. Andreychin” Bulgarian Academy of Sciences svetla@dcl.bas.bg Abstract In this paper, we present work in progress aimed at the development of a new image dataset with annotated objects. The Multilin- gual Image Corpus consists of an ontology of visual objects (based on WordNet) and a collection of thematically related images an- notated with segmentation masks and object classes. We identified 277 dominant classes and 1,037 parent and attribute classes, and grouped them into 10 thematic domains such as sport, medicine, education, food, security, etc. For the selected classes a large-scale web image search is being conducted in or- der to compile a substantial collection of high- quality copyright free images. The focus of the paper is the annotation protocol which we established to facilitate the annotation process: the Ontology of visual objects and the conven- tions for image selection and for object seg- mentation. The dataset is designed both for image classification and object detection and for semantic segmentation. In addition, the object annotations will be supplied with mul- tilingual descriptions by using freely available wordnets. 1 Introduction We are surrounded by information represented by text, images and video data in multimodal streams. One of the processing tasks for large multimodal data streams is automatic image description (image classification, object segmentation and classifica- tion), which is directly connected with the task of image search, as well as with the expansion of the scope for automatic question answering regarding images. The goal of our project Multilingual Image Corpus (MIC 21)1 is to provide a large image dataset with annotated objects and object descrip- tions in (at least) 20 European languages. The 1https://dcl.bas.bg/mic21/ Multilingual Image Corpus consists of an ontol- ogy of visual objects (based on WordNet) and a collection of thematically related images whose ob- jects are annotated with segmentation masks and labels describing the ontology classes. The dataset is designed both for image classification and object detection and for semantic segmentation. The main contributions of our work are: a) the provision of large collection of high-quality copy- right free images; b) the formulation of the On- tology of visual objects based on WordNet noun hierarchies; c) the precise manual correction of automatic object segmentation within the images and the annotation of object classes; and d) the association of objects and images with extended multilingual descriptions based on WordNet inner- and interlingual relations. We have divided the annotation process into four main stages: a) definition of an ontology of vi- sual objects; b) collection of appropriate images; c) automatic object segmentation; and d) manual cor- rection of object segmentation and manual classifi- cation of objects. The annotation protocol includes the Ontology of visual objects and the conventions for image selection and for object segmentation. The focus of the paper is the annotation protocol which is established to facilitate the manual annota- tion. We begin …

Content truncated to 500 words