Finding Definitions in Large Corpora with Sketch Engine

Finding Definitions in Large Corpora with Sketch Engine Vojtˇech Kov´aˇr1,2, Monika Moˇciarikov´a1, Pavel Rychl´y1,2 1NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic 2Lexical Computing CZ, Brno, Czech Republic xkovar3@fi.muni.cz, 374485@mail.muni.cz, pary@fi.muni.cz Abstract The paper describes automatic definition finding implemented within the leading corpus query and management tool, Sketch Engine. The implementation exploits complex pattern-matching queries in the corpus query language (CQL) and the indexing mechanism of word sketches for finding and storing definition candidates throughout the corpus. The approach is evaluated for Czech and English corpora, showing that the results are usable in practice: precision of the tool ranges between 30 and 75 percent (depending on the major corpus text types) and we were able to extract nearly 2 million definition candidates from an English corpus with 1.4 billion words. The feature is embedded into the interface as a concordance filter, so that users can search for definitions of any query to the corpus, including very specific multi-word queries. The results also indicate that ordinary texts (unlike explanatory texts) contain rather low number of definitions, which is perhaps the most important problem with automatic definition finding in general. Keywords: Sketch Engine, definition, definitions, CQL, corpora 1. Introduction Definitions, in the sense of text descriptions of particular concepts, are an important part of any learning material, and especially dictionaries. At present, definitions in such materials are always hand-made, created by specialists in the particular field, or professional lexicographers. This situation has two significant drawbacks – first, think- ing up and writing the definitions is quite expensive; a specialist, or an educated lexicographer is needed for this work, and it is also quite an abstract and time-consuming task. Second, the knowledge of the particular specialist, and/or the time for thinking about the definition, may be limited and therefore the created definitions may not be good enough. This paper presents a partial solution to these problems. From big data, in form of huge text corpora, we automat- ically extract sentences that may contain definitions and present them to a user (lexicographer/specialist) working on a particular concept; then the user has a possibility to simply re-use a particular definition, or adjust one of them according to others. The process of thinking about defini- tions is then made faster and more straightforward. The work presented here is a rule based pattern matching approach developed and integrated within the Sketch En- gine corpus query system (Kilgarriff et al., 2014a), primar- ily dedicated to lexicographers, so we are aiming mainly at lexicographic type of definitions. The whole exercise fits into a larger frame of streamlining production of dictionar- ies; after automatic terminology extraction (Kilgarriff et al., 2014b) finding typical collocations (Kilgarriff et al., 2004) and educational examples (Rychl´y et al., 2008), definition creation is one of the last parts of the process without sig- nificant computational support. In the paper, we briefly mention related work done on this task, then describe our implementation within the Sketch Engine, and provide a precision evaluation of this approach, …

Content truncated to 500 words