ComboNER: A Lightweight All-In-One POS Tagger, Dependency Parser and NER

Proceedings of Recent Advances in Natural Language Processing, pages 1508–1514 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_169 1508 ComboNER: A Lightweight All-In-One POS Tagger, Dependency Parser and NER Aleksander Wawer Institute of Computer Science, Polish Academy of Sciences Jana Kazimierza 5 01-248 Warszawa, Poland axw@ipipan.waw.pl Γ orcid.org/0000-0002-7081-9797 Abstract The current natural language processing is strongly focused on raising accuracy. The progress comes at a cost of super-heavy mod- els with hundreds of millions or even bil- lions of parameters. However, simple syntac- tic tasks such as part-of-speech (POS) tagging, dependency parsing or named entity recogni- tion (NER) do not require the largest models to achieve acceptable results. In line with this assumption we try to minimize the size of the model that jointly performs all three tasks. We introduce ComboNER: a lightweight tool, or- ders of magnitude smaller than state-of-the- art transformers. It is based on pre-trained subword embeddings and recurrent neural net- work architecture. Polish language data.ComboNER operates on The model has out- puts for POS tagging, dependency parsing and NER. Our paper contains some insights from fine-tuning of the model and reports its overall results. 1 Introduction The paradigm for automated language processing was originally based on building dedicated tools for every task. We illustrate it on the example of the Polish language, but it is very likely the case also for other languages. The dedicated tools in- clude morphological taggers and disambiguators (a Polish language approximation of POS tagging), dedicated dependency parsers and named entity recognizers. For convenience, the tools sometimes were made available as web services and possi- bly composed into chains. One such example is Multiservice (Ogrodniczuk and Lenart, 2012), a linguistic Web service for Polish, combining sev- eral mature offline linguistic tools in a common online platform. Multiservice offers a number of pre-defined chains. For example, the dependency parser requires morphological tagging and disam- biguation as the initial step. The two tools are put together in appropriate order in one chain. Among universal tools (not dedicated to the Pol- ish language) it is worth to mention: • UDPipe, a pipeline (tool) for tokenization, tag- ging, lemmatization and dependency parsing (Straka et al., 2016). • NLP-Cube which in addition to the capabil- ities mentioned for UDPipe offers sentence segmentation and lemmatization. The re- ported results cover many languages but not Polish (Boros, et al., 2018) • Stanza, a tool for tokenization, multi-word token (MWT) expansion, lemmatization, part- of-speech (POS) and morphological features tagging, dependency parsing, and named en- tity recognition. Named entity recognition models are separate and Polish is not avail- able (Qi et al., 2020). Recently, with the advances of neural networks in natural language processing, new tools were pro- posed that simplify the chains by doing several tasks at once. One example is Combo (Rybak and Wr´oblewska, 2018), a bi-directional recurrent neu- ral network (RNN) using word2vec embeddings with multiple outputs for morphological and depen- dency analysis. The model was submitted to the CoNLL 2018 Universal Dependency shared task. The Combo tool, while doing certain things …

Content truncated to 500 words