Improving corpus search via parsing

Improving Corpus Search via Parsing Natalia Klyueva and Pavel Straˇnák Charles University in Prague, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Malostranské námˇestí 25, 118 00, Prague 1, Czech Republic {kljueva,stranak}@ufal.mff.cuni.cz Abstract In this paper, we describe an addition to the corpus query system Kontext that enables to enhance the search using syntactic attributes in addition to the existing features, mainly lemmas and morphological categories. We present the enhancements of the corpus query system itself, the attributes we use to represent syntactic structures in data, and some examples of querying the syntactically annotated corpora, such as treebanks in various languages as well as an automatically parsed large corpus. Keywords: corpus query, treebanks, parsed corpora, syntactic search 1. Introduction Traditionally, corpus search engines like SketchEngine1, IMS Corpus Workbench2 and other search in a linear rep- resentation of a sentence as "attributed strings". They dis- play results of the searches as concordances. In comparison to that treebank search engines like PML-TQ3(Pajas et al., 2009) or INESS4 or Tundra5 allow sophisticated queries for the tree structures of treebanks and they typically display results as trees (one at a time). Their query languages are more expressive, but necessarily also more complicated. In this paper we present a compromise that we believe can be useful in many situations: adding limited capabilities for syntactic search to the corpus search engine Kontext6 with its CQL language and adding display of syntactic trees of sentences to the concordance view of the search results in Kontext. In the LINDAT/CLARIN centre7 we use the Kontext search engine as our main tool to allow search in our corpora, with a long-term goal to provide search for all the corpora avail- able in the LINDAT/CLARIN repository8. We also provide a specialised treebank search tool PML Tree Query (PML-TQ) and currently about 70 treebanks are available for search there. However PML-TQ as well as other similar tools do not display concordances, they can- not efficiently process parsed corpora orders of magnitude larger than manually created treebanks and so on. For these reasons we believe that adding syntactic search, even if lim- ited, and display of syntactic trees to Kontext may provide useful complement to the PML-TQ functionality not only for large datasets, but also for treebanks like Prague Depen- dency Treebank (Bejˇcek et al., 2013), HamleDT (Zeman et al., 2015) or Universal Dependencies (Nivre et al., 2015), 1http://sketchengine.co.uk 2http://cwb.sourceforge.net 3http://hdl.handle.net/11858/ 00-097C-0000-0022-C7F6-3 4http://iness.uib.no 5http://weblicht.sfs.uni-tuebingen.de/ Tundra/ 6https://github.com/czcorpus/kontext 7http://lindat.cz 8https://lindat.mff.cuni.cz/repository/ for which it provides complementary interface and abilities. 2. Lindat corpora In Lindat repository, there are 648 items of the type "cor- pus", but most of them does not contain files, just a metain- formation. We feature 88 items that contain corpora, and our main aim is to load all of them to KonText with the ex- ception of those that are already available in the Czech Na- tional Corpus via the same search engine. We plan to parse the corpora that do not have syntactic annotation in case we have the …

Content truncated to 500 words