A Web-based Collaborative Annotation and Consolidation Tool

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7053–7059 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7053 A Web-based Collaborative Annotation and Consolidation Tool Tobias Daudert Insight Centre for Data Analytics National University of Ireland, Galway tobias.daudert@insight-centre.org Abstract Annotation tools are a valuable asset for the construction of labelled textual datasets. However, they tend to have a rigid structure, closed back-end and front-end, and are built in a non-user-friendly way. These downfalls difficult their use in annotation tasks requiring varied text formats, prevent researchers to optimise the tool to the annotation task, and impede people with little program- ming knowledge to easily modify the tool rendering it unusable for a large cohort. Targeting these needs, we present a web-based collaborative annotation and consolidation tool (AWOCATo), capable of supporting varied textual formats. AWOCATo is based on three pillars: (1) Simplicity, built with a modular architecture employing easy to use technologies; (2) Flexibility, the JSON configuration file allows an easy adaption to the annotation task; (3) Customizability, parameters such as labels, colours, or consolidation features can be easily customized. These features allow AWOCATo to support a range of tasks and domains, filling the gap left by the absence of annotation tools that can be used by people with and without programming knowledge, including those who wish to easily adapt a tool to less common tasks. AWOCATo is available for download at https://github.com/TDaudert/AWOCATo. Keywords: annotation, consolidation, curation, tool, text, textual annotation, sentiment analysis, classification 1. Introduction The continuous rise in data enabled new opportunities in computer science (Hey and Trefethen, 2003). Large vol- umes of data can be used in machine learning approaches for a variety of tasks such as text classification (Khan et al., 2010), image recognition (Joutou and Yanai, 2009), and disease prediction (Chen et al., 2017). Labelled datasets are fundamental for the evaluation of these models and for training on supervised approaches, however, they tend to be sparse (Jeong et al., 2009; He et al., 2007). Annotation is known to be a complex task, involving many people and stages (Finlayson and Erjavec, 2017); as aid, researchers can leverage annotation tools. Albeit facilitating and im- proving the efficiency of annotation tasks, annotation tools tend to have a rigid structure and a complex interface, thus, requiring a steep learning curve and discouraging the de- ployment of tailored tasks. In addition, this complexity de- ters potential users without programming skills to employ such tools in their annotations task. To fulfil these needs, we present a web-based/online col- laborative annotation and consolidation tool (in short, AWOCATo), capable of supporting varied textual formats. AWOCATo’s structure is based on three pillars: 1. Simplicity - built with Python and utilising Mon- goDB1 as back-end. 2. Flexibility - annotation and curation task features are configurable in a JSON2 file. 3. Customizability - fully adaptable front-end coded with HTML, CSS, and Javascript. In this paper, we will deconstruct AWOCATo providing details on the back-end architecture, front-end, and, most …

Content truncated to 500 words