Facilitating Corpus Usage: Making Icelandic Corpora More Accessible for Researchers and Language Users

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3399–3405 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3399 Facilitating Corpus Usage: Making Icelandic Corpora More Accessible for Researchers and Language Users Steinþór Steingrímsson, Starkaður Barkarson, Gunnar Thor Örnólfsson The Árni Magnússon Institute for Icelandic Studies steinthor.steingrimsson@arnastofnun.is, starkadur.barkarson@arnastofnun.is, gunnar.thor.ornolfsson@arnastofnun.is Abstract We introduce an array of open and accessible tools to facilitate the use of the Icelandic Gigaword Corpus, in the field of Natural Language Processing as well as for students, linguists, sociologists and others benefitting from using large corpora. A KWIC engine, powered by the Swedish Korp tool is adapted to the specifics of the corpus. An n-gram viewer, highly customizable to suit different needs, allows users to study word usage throughout the period of our text collection. A frequency dictionary provides much sought after information about word frequency statistics, computed for each subcorpus as well as aggregate, disambiguating homographs based on their respective lemmas and morphosyntactic tags. Furthermore, we provide n-grams based on the corpus, and a variety of pre-trained word embeddings models, based on word2vec, GloVe, fastText and ELMo. For three of the model types, multiple word embedding models are available trained with different algorithms and using either lemmatised or unlemmatised texts. Keywords: Icelandic, Corpora, KWIC, ngrams, frequency, word embeddings 1. Introduction The need for large text corpora has become increasingly urgent in recent years. As data-oriented methods have come to dominate the field of Natural Language Processing (NLP), and in order to achieve better performance, larger datasets have to be compiled. This is especially important with the rise in popularity of neural networks and various word embedding techniques, such as word2vec (Mikolov et al., 2013a; Mikolov et al., 2013b), GloVe (Pennington et al., 2014), fastText (Bojanowski et al., 2017; Joulin et al., 2017) and contextual embeddings like ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019). Large corpora are also useful for other fields of research. Corpus-based linguistics is a fast-growing methodology in linguistics (see e.g. Gries, 2009), used prominently in syn- tax research, but also in other fields of linguistics like se- mantics, morphology and phonology. Lexicographers have for a long time used corpora in one form or another, mostly in the form of citations, but since the first dictionary based on a specific corpus, Collins COBUILD, was published in 1987 (Sinclair et al., 1987), many others have taken a sim- ilar path. In order to use corpora effectively in their work, modern lexicographers need powerful tools. They need to be able to discern between word senses, access actual us- age examples, research origins of neologisms and study frequency data on word usage or usage of multiword ex- pressions, to name a few examples. A good corpus tool is key to a comprehensive lexicographic analysis – a corpus without a good tool to access it is of little use (Kilgarriff and Kosem, 2012). Michel et al. (2011) established a method of study they coined culturomics …

Content truncated to 500 words