PyVallex: A Processing System for Valency Lexicon Data

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 7187–7193 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 7187 PyVallex: A Processing System for Valency Lexicon Data Jonathan Verner1,2, Anna Vernerov´a1 1: Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics 2: Charles University, Faculty of Arts, Department of Logic 1: Malostransk´e n´amˇest´ı 25, 118 00 Prague 1, Czech Republic; 2: Celetn´a 20, 110 00 Prague 1, Czech Republic jonathan@temno.eu, vernerova@ufal.mff.cuni.cz Abstract We present PyVallex, a Python-based system for presenting, searching/filtering, editing/extending and automatic processing of machine- readable lexicon data originally available in a text-based format. The system consists of several components: a parser for the specific lexicon format used in several valency lexicons, a data-validation framework, a regular expression based search engine, a map-reduce style framework for querying the lexicon data and a web-based interface integrating complex search and some basic editing capabilities. PyVallex provides most of the typical functionalities of a Dictionary Writing System (DWS), such as multiple presentation modes for the underlying lexical database, automatic evaluation of consistency tests, and a mechanism of merging updates coming from multiple sources. The editing functionality is currently limited to the client-side interface and edits of existing lexical entries, but additional script-based operations on the database are also possible. The code is published under the open source MIT license and is also available in the form of a Python module for integrating into other software. Keywords: dictionary management, lexicographic software, electronic dictionaries, valency lexicons 1. When The Old Tools No Longer Suffice In the early 2000s researchers at the Institute of Formal and Applied Linguistics in Prague (then the Center for Com- putational Linguistics) started developing the valency lex- icon Vallex aiming to validate the existing valency the- ory of the Functional Generative Description (Panevov´a, 1974; Panevov´a, 1975; Panevov´a, 1980; Panevov´a and Sk- oumalov´a, 1992) and to pursue further syntactic research (Lopatkov´a et al., 2002). Early on, a lexicon named PDT- Vallex was forked from Vallex and used for maintaining consistency of the tectogrammatical annotation of the data in the Prague Dependency Treebank 2.0 (Hajiˇc et al., 2006; Hajiˇc and Honetschl¨ager, 2003). The PDT-Vallex was fully integrated with the tree editor and search tool TrEd1 (Pajas and ˇStˇep´anek, 2008) and its native format is now XML- based. In contrast, the Vallex project still uses the orig- inal pipeline based on a text-based format converted by a sequence of scripts to an intermediate representation as XML-data (ˇZabokrtsk´y, 2005) and final presentations as web-pages (with individual lexicon entries stored in html files) and a PDF/printed lexicon (produced via TeX). The annotators are reluctant to move away from the text-based format which they can use with a common text-editor (with custom syntax highlighting definition files relying on a set of markup conventions); this situation is not unique to the Vallex project, e.g. (Benko, 2019). On the other hand, the set of custom scripts for format conversions, validity check- ing, and custom …

Content truncated to 500 words