Bye, Bye, Maintenance Work? Using Model Cloning to Approximate the Behavior of Legacy Tools

Bye, Bye, Maintenance Work? Using Model Cloning to Approximate the Behavior of Legacy Tools Piush Aggarwal Torsten Zesch Computational Linguistics CATALPA - Center for Advanced Technology-Assisted Learning and Predictive Analytics FernUniversit¨at in Hagen {piush.aggarwal,torsten.zesch}@fernuni-hagen.de Abstract A lot of NLP tools are not maintained anymore, but might still provide some unique functional- ity. We investigate whether such legacy tools could be replaced by a neural network that closely imitates the original behavior. For this purpose, we propose model cloning that can be performed by solely looking at the output of the original model, which makes the cloning possi- ble also for black-box systems. Using a single neural architecture for cloning legacy models, caries other benefits like ease-of-use, continued maintenance, and expected speed increase. As a proof-of-concept, we clone 9 models from 5 POS tagger implementations of different com- plexity. The cloned models all learn to perform POS tagging on par with the legacy models, but seem not to learn the specific tagging patterns of individual legacy models. 1 Introduction End-to-end neural models are increasingly used to build NLP tools (Tao et al., 2022; Wolf et al., 2020; Qi et al., 2020; Akbik et al., 2019; Han et al., 2019) However, legacy tools are still being used in pro- duction and for research purposes, as they might provide a unique functionality that cannot be easily replaced. Such legacy tools are often not main- tained anymore and increasingly hard to use. Or outright dangerous, as the Log4Shell vulnerability1 has turned some legacy Java tools into unmanage- able security risks. They might only work with a specific OS version or with an outdated version of the programming language. Or the required models have to be secretly traded between researchers, as the official download ceased to exist. For some very important tools, it might be possible to port them to the latest technology and keep them available, but the bulk of legacy tools will soon be gone. 1https://en.wikipedia.org/wiki/Log4Shell Figure 1: The model cloning process. We argue that a possible solution is to clone the legacy models into a state-of-the-art neural model. We consider here a situation where the original training data is not available. Otherwise, we could simply retrain the model. The legacy models might also include hard-coded heuristics or dictionaries that are not reflected in the training data itself. We thus propose to apply the legacy model on plain text and then use the results to train a new model.2 In this paper, we choose POS tagging as a proof- of-concept use case to illustrate the potential prop- erties of model cloning. We choose 9 different POS models from 5 legacy tools and clone their behavior into BiLSTM-CRF networks (Huang et al., 2015; Graves et al., 2013). We make all of the generated cloned models and our experimental code publicly available.3 2 Model Cloning Under model cloning, we understand the process of copying the behavior of a legacy model by only looking at its output. Figure 1 gives an overview of the cloning process, …

Content truncated to 500 words