A New Latin Treebank for Universal Dependencies: Charters between Ancient Latin and Romance Languages

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 90–98 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 90 Cross-lingual Zero Pronoun Resolution Abdulrahman Aloraini1,2, Massimo Poesio1 1School of Electronic Engineering and Computer Science, Queen Mary University of London 2Department of Information Technology, Qassim University {a.aloraini, m.poesio}@qmul.ac.uk Abstract In languages like Arabic, Chinese, Italian, Japanese, Korean, Portuguese, Spanish, and many others, predicate arguments in certain syntactic positions are not realized instead of being realized as overt pronouns, and are thus called zero- or null-pronouns. Identifying and resolving such omitted arguments is crucial to machine translation, information extraction and other NLP tasks, but depends heavily on semantic coherence and lexical relationships. We propose a BERT-based cross-lingual model for zero pronoun resolution, and evaluate it on the Arabic and Chinese portions of OntoNotes 5.0. As far as we know, ours is the first neural model of zero-pronoun resolution for Arabic; and our model also outperforms the state-of-the-art for Chinese. In the paper we also evaluate BERT feature extraction and fine-tune models on the task, and compare them with our model. We also report on an investigation of BERT layers indicating which layer encodes the most suitable representation for the task. Our code is available at https://github.com/amaloraini/cross-lingual-ZP. Keywords: zero pronouns, cross-lingual classification, anaphora resolution 1. Introduction In pronoun-dropping (pro-drop) languages such as Arabic (Eid, 1983), Chinese (Li and Thompson, 1979), Italian (Di Eugenio, 1990) and other romance languages (e.g., Por- tuguese, Spanish), Japanese (Kameyama, 1985), and others (Kim, 2000), arguments can be elided in certain contexts in which a pronoun is used in English, such as subjects. We use the term zero-pronouns (ZP) to refer to these un- realised arguments, the most used in the recent literature.1 Anaphoric zero-pronoun (AZP) are zero-pronouns that re- fer to one or more noun phrases that appear previously in a text. The following example of an AZP comes from the Arabic section of OntoNotes: ﺑﯿﺎناﻟﺤﺮﯾﺮيﺗﻤﯿﺰﺑﺘﻔﺎﺻﯿﻞ...ﺣﯿﺚرﻛﺰ*ﻋﻠﻰأنﻣﺠﻠﺲ وزراءﻟﺒﻨﺎنوﺣﺪهاﻟﻤﺴﺆولﻋﻦ... Alhariri’s statement included more details ...in which (he) empha- sized that the council of ministers of Lebanon is the only represen- tative ... In the example, the zero pronoun indicated with ’*’ refers to an entity introduced with a masculine singular noun that was previously mentioned in the sentence. (In OntoNotes 5.0, zero pronouns are denoted as * in Arabic text, and *pro* in Chinese). AZP resolution usually consists of two steps: extracting ZPs that are anaphoric, and identifying the correct antecedents for AZPs. Our focus is on the latter because there has been no proposal for Arabic. In this paper we propose a cross- lingual, BERT based model of zero pronoun resolution. Our contributions include: • We propose a novel cross-lingual, BERT-based model and test it on languages that differ completely in their morphological structure: Arabic and Chinese. (Arabic is morphologically rich, whereas Chinese’s morphol- ogy is relatively simple (Pradhan et al., 2012)) • As far as we know this is the first neural network-based ZP resolution model for Arabic, and …

Content truncated to 500 words