TIARA: A Tool for Annotating Discourse Relations and Sentence Reordering

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6912–6920 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6912 TIARA: A Tool for Annotating Discourse Relations and Sentence Reordering Jan Wira Gotama Putra†, Simone Teufel‡†, Kana Matsumura§, Takenobu Tokunaga† †Tokyo Institute of Technology, Japan ‡University of Cambridge, United Kingdom §Waseda University, Japan gotama.w.aa@m.titech.ac.jp, simone.teufel@cl.cam.ac.uk, kmatsumura@suou.waseda.jp, take@c.titech.ac.jp Abstract This paper introduces TIARA, a new publicly available web-based annotation tool for discourse relations and sentence reordering. Annotation tasks such as these, which are based on relations between large textual objects, are inherently hard to visualise without either cluttering the display and/or confusing the annotators. TIARA deals with the visual complexity during the annotation process by systematically simplifying the layout, and by offering interactive visualisation, including coloured links, indentation, and dual-view. TIARA’s text view allows annotators to focus on the analysis of logical sequencing between sentences. A separate tree view allows them to review their analysis in terms of the overall discourse structure. The dual-view gives it an edge over other discourse annotation tools and makes it particularly attractive as an educational tool (e.g., for teaching students how to argue more effectively). As it is based on standard web technologies and can be easily customised to other annotation schemes, it can be easily used by anybody. Apart from the project it was originally designed for, in which hundreds of texts were annotated by three annotators, TIARA has already been adopted by a second discourse annotation study, which uses it in the teaching of argumentation. Keywords: annotation tool, discourse annotation, discourse relations, sentence reordering, teaching of argumentation 1. Introduction There are many important aspects in writing such as gram- mar, mechanics, writing style and coherence (Grosz and Sidner, 1986; Lee and Webster, 2012). Out of these as- pects, textual coherence, as an aspect of discourse structure, is extremely important. It concerns how sentences or other discourse units form a flow of meaning (Grosz and Sidner, 1986), and has been extensively analysed in the past (Ja- cobs et al., 1981; Grosz et al., 1995; Mann and Thompson, 1988; Wolf and Gibson, 2005; Garing, 2014). Our long- term goal is to automatically reorder sentences in argumen- tative essays so that their coherence is improved, as well as providing an explanation for the changes made. Therefore, the analysis of the order of sentence in original texts (which we call first drafts) and in their improved versions is neces- sary. A tool is needed to support these annotations, and we believe that it can be designed in such a way that it is useful for language and argument education in general. Existing theory of text coherence stipulate that the order of sentences mirrors the intentional structure of discourse (Grosz and Sidner, 1986). In our situation, however, where we are dealing with often badly-structured first drafts, the intentional and surface structure are often not well-aligned. There are many ways how the isomorphism could be re- stored, leading to an improved …

Content truncated to 500 words