Data Augmentation using Machine Translation for Fake News Detection in the Urdu Language

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 236–244 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 236 Crowdsourcing in the Development of a Multilingual FrameNet: A Case Study of Korean FrameNet Younggyun Hahm1, Youngbin Noh1, Jiyoon Han2, Taehwan Oh2, Hyonsu Choe2, Hansaem Kim2, Key-Sun Choi1 1KAIST, 2Yonsei University {hahmyg, vincenoh, kschoi}@kaist.ac.kr, {clinamen35, ghksl0604, choehyonsu, khss}@yonsei.ac.kr Abstract Using current methods, the construction of multilingual resources in FrameNet is an expensive and complex task. While crowdsourcing is a viable alternative, it is difficult to include non-native English speakers in such efforts as they often have difficulty with English-based FrameNet tools. In this work, we investigated cross-lingual issues in crowdsourcing approaches for multilingual FrameNets, specifically in the context of the newly constructed Korean FrameNet. To accomplish this, we evaluated the effectiveness of various crowdsourcing settings whereby certain types of information are provided to workers, such as English definitions in FrameNet or translated def- initions. We then evaluated whether the crowdsourced results accurately captured the meaning of frames both cross-culturally and cross-linguistically, and found that by allowing the crowd workers to make intuitive choices, they achieved a quality comparable to that of trained FrameNet experts (F1 > 0.75). The outcomes of this work are now publicly available as a new release of Korean FrameNet 1.1. Keywords: FrameNet, Multilingual, Crowdsourcing 1. Introduction FrameNet is a large linguistic resource in which the meaning of text is represented using Frame Semantics (frames) (Baker et al., 1998; Fillmore et al., 2003). An English language version of FrameNet was created by the FrameNet Project at the International Computer Science In- stitute1 (ICSI), which defined English frames and assigned them to English vocabularies in order to construct lexical units (LUs) consisting of a word and frame pair. During an- notation, the process of which was formalized by Ruppen- hofer et al. (2006), annotators chose a proper frame (fram- ing) for each target word and corresponding frame elements for the arguments. Since being developed, FrameNet has become a useful resource in a variety of fields, and re- searchers are now showing increasing interest in not only English FrameNet, but also in multilingual FrameNets for other languages (Baker et al., 2018; Borin et al., 2010; Meurs et al., 2008; sub, 2003; Burchardt et al., 2006; Yang et al., 2018; Virk and Prasad, 2018). However, despite this interest, the construction of multilingual FrameNets re- mains challenging due to the expense and complexity of annotating over raw sentences, including those in English. As an alternative, crowdsourcing can be used to construct multilingual FrameNets. Several studies have investigated the viability of constructing FrameNets via crowdsourcing and found that annotations obtained via a crowd of 10 peo- ple are reliable when compared to annotations provided by experts (Fossati et al., 2013; Dumitrache et al., 2018). Sim- ilar studies have also been conducted for non-English lan- guages. For example, Ohara et al. (2018) employed a Kyoto case frame dictionary when performing framing via crowd- sourcing on …

Content truncated to 500 words