QTLeap WSD/NED Corpora: Semantic Annotation of Parallel Corpora in Six Languages

QTLeap WSD/NED Corpora: Semantic Annotation of Parallel Corpora in Six Languages Arantxa Otegi,∗ Nora Aranberri∗, Antonio Branco‡, Jan Hajiˇc†, Steven Neale‡, Petya Osenova ∇, Rita Pereira‡, Martin Popel†, Jo˜ao Silva‡, Kiril Simov∇ and Eneko Agirre∗ ∗ University of the Basque Country (UPV/EHU), IXA Group – {arantza.otegi, nora.aranberri, e.agirre}@ehu.eus † Charles University in Prague, Faculty of Mathematics and Physics, UFAL – {hajic, popel}@ufal.mff.cuni.cz ‡Universidade de Lisboa – {antonio.branco, steven.neale, ana.pereira, jsilva}@di.fc.ul.pt ∇Institute of Information and Communication Technologies (IICT-BAS) – {petya, kivs}@bultreebank.org Abstract This work presents parallel corpora automatically annotated with several NLP tools, including lemma and part-of-speech tagging, named-entity recognition and classification, named-entity disambiguation, word-sense disambiguation, and coreference. The corpora comprise both the well-known Europarl corpus and a domain-specific question-answer troubleshooting corpus on the IT domain. English is common in all parallel corpora, with translations in five languages, namely, Basque, Bulgarian, Czech, Portuguese and Spanish. We describe the annotated corpora and the tools used for annotation, as well as annotation statistics for each language. These new resources are freely available and will help research on semantic processing for machine translation and cross-lingual transfer. Keywords: annotated parallel corpora, named-entity disambiguation, word sense disambiguation, coreference 1. Introduction From a machine translation (MT) perspective, the deeper the processing of utterances, the less language-specific dif- ferences will remain between the representations of the meaning of the source and target texts. As a result, chances of success are expected to increase considerably by MT systems that are based on deeper semantic engineering ap- proaches. Following this assumption, one of the approaches taken by the QTLeap project1 is to enrich MT training re- sources with lexico-semantic information. In this work, we present a solid effort to build multilin- gual parallel corpora annotated at multiple semantic lev- els. Our overall goal is to enrich two parallel corpora, Eu- roparl (Koehn, 2005) and the QTLeap corpus (Agirre et al., 2015b), with token, lemma, part-of-speech (POS), named- entity recognition and classification (NERC), named-entity disambiguation (NED), word-sense disambiguation (WSD) and coreference for six languages covered in the QTLeap project, namely, Basque (EU), Bulgarian (BG), Czech (CS), English (EN), Portuguese (PT) and Spanish (ES). Specifically, this paper presents the first release of such cor- pora, which includes NERC, NED, WSD and coreference- level annotation for these six languages. Additionally, some languages have extra annotations, such as wikifica- tion (EN, ES), dependency parsing (BG, CS, EU) or con- stituency parsing (EN, ES), and semantic-roles (EN, ES). The annotated “Europarl-QTLeap WSD/NED corpus” and “QTLeap WSD/NED corpus” are distributed under the li- cense CC BY-NC-SA 4.0, and have been released through Meta-share2 and CLARIN Lindat.3 1http://qtleap.eu 2http://metashare.metanet4u.eu/go2/ europarl-qtleap-wsdned-corpus http://metashare.metanet4u.eu/go2/ qtleap-wsdned-corpus 3https://lindat.mff.cuni.cz, namely: http://hdl.handle.net/11234/1-1476 This paper is organized as follows: Section 2 presents the corpora we annotated; Section 3 describes the NERC, WSD, NED and coreference tools and annotation formats; Section 4 addresses the evaluation of the tools; Section 5 presents the corpus statistics of the annotation; and finally Section 6 outlines the conclusions. 2. Target Corpora This section describes the two corpora we have annotated, …

Content truncated to 500 words