Embeddings for Named Entity Recognition in Geoscience Portuguese Literature

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4625–4630 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4625 Embeddings for Named Entity Recognition in Geoscience Portuguese Literature Bernardo Consoli1, Joaquim Santos1, Diogo Gomes2, Fabio Cordeiro2, Renata Vieira1 and Viviane Moreira3 1Pontifical Catholic University of Rio Grande do Sul, 2Petrobras Research and Development Center, 3Federal University of Rio Grande do Sul {Porto Alegre1,3, Rio de Janeiro2}, Brazil {bernardo.consoli, joaquim.santos}@acad.pucrs.br, {diogo.gomes, fabio.cordeiro}@petrobras.com.br, renata.vieira@pucrs.br, viviane@inf.ufrgs.br Abstract This work focuses on Portuguese Named Entity Recognition (NER) in the Geology domain. The only domain-specific dataset in the Portuguese language annotated for Named Entity Recognition is the GeoCorpus. Our approach relies on Bidirecional Long Short-Term Memory - Conditional Random Fields neural networks (BiLSTM-CRF) - a widely used type of network for this area of research - that use vector and tensor embedding representations. We used three types of embedding models (Word Embeddings, Flair Embeddings, and Stacked Embeddings) under two versions (domain-specific and generalized). We originally trained the domain specific Flair Embeddings model with a generalized context in mind, but we fine-tuned with domain-specific Oil and Gas corpora, as there simply was not enough domain corpora to properly train such a model. We evaluated each of these embeddings separately, as well as we stacked with another embedding. Finally, we achieved state-of-the-art results for this domain with one of our embeddings, and we performed an error analysis on the language model that achieved the best results. Furthermore, we investigated the effects of domain-specific versus generalized embeddings. Keywords: Geology, Named Entity Recognition, Word Embeddings, Stacked Embeddings 1. Introduction Named Entity Recognition (NER) is a task within the field of Natural Language Processing (NLP) that deals with the identification and categorization of Named Entities (NE) in a given text. Resolutions to this task usually focus on a conventional set of categories, with the most commonly used being the Person, Location and Organization cate- gories (dos Santos and Guimar˜aes, 2015). The Geology domain, commonly defined as the science that studies the origin, history, life, and structure of the Earth, must be described by other than the conventional Named Entity Recognition categories. While the Person, Location, and Organization categories can indeed be found within corpora of this domain, they, and other conventional cat- egories, do not account for the vast majority of Named En- tities found within such texts. There is still a scarcity of research focusing on this domain, on the other hand the domain has a potentially large quan- tity of non-conventional Named Entities categories, in fact, yet to be defined. There is little consensus in literature as to what these categories should be to represent the core of this domain. In this work we consider the subdomain of Brazilian Sedimentary Basins, and defined the categories accordingly. For the Named Entity Recognition problem, in gen- eral, Neural Networks (NN) coupled with Word Embed- dings (WE) are usually present in the best evaluated sys- tems (Lample et al., 2016; dos Santos …

Content truncated to 500 words