Few-Shot Tabular Data Enrichment Using Fine-Tuned Transformer Architectures

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 1577 - 1591 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Few-Shot Tabular Data Enrichment Using Fine-Tuned Transformer Architectures Asaf Harari, Gilad Katz Ben-Gurion University of the Negev, P.O.B. 653 Beer-Sheva, Israel {hsaf,giladkz}@post.bgu.ac.il Abstract The enrichment of tabular datasets using ex- ternal sources has gained significant attention in recent years. Existing solutions, however, either ignore external unstructured data com- pletely or devise dataset-specific solutions. In this study, we proposed Few-Shot Transformer based Enrichment (FeSTE), a generic and ro- bust framework for the enrichment of tabular datasets using unstructured data. By training over multiple datasets, our approach is able to develop generic models that can be applied to additional datasets with minimal training (i.e., few-shot). Our approach is based on an adapta- tion of BERT, for which we present a novel fine- tuning approach that reformulates the tuples of the datasets as sentences. Our evaluation, conducted on 17 datasets, shows that FeSTE is able to generate high quality features and significantly outperform existing fine-tuning solutions. 1 Introduction Tabular data is the most diverse format of data rep- resentation, spanning domains from nutrition to banking. It does, however, suffer from a lack of contextual information that could make its analysis more effective. Data scientists seek to overcome this limitation by using feature engineering (FE), which involves applying transformations on exist- ing features to create additional representations of the data. When the available data is not suffi- ciently diverse (or when additional improvement is sought), one may attempt to use external infor- mation sources to enrich the data. We refer to this process as external enrichment of datasets (EED). The use of external sources for feature engineer- ing is both computationally-heavy and time con- suming. The process first involves matching en- tities in the data to those in the external source, a process known as Entity Linking (Shen et al., 2014). Once entities in the external source have been matched, candidate features need to be gener- ated, evaluated, and finally integrated into the tabu- lar dataset. While multiple studies in recent years (Paulheim and Fümkranz, 2012; Ristoski et al., 2015; Friedman and Markovitch, 2018; Mountan- tonakis and Tzitzikas, 2017; Galhotra et al., 2019; Harari and Katz, 2022) have sought to automate the EED process, a large majority focuses solely on structured external sources, e.g., DBpedia ta- bles, and do not attempt to use the large amounts of available unstructured data (i.e., free text). In this study, we present Few-Shot Transformer based Enrichment (FeSTE) a generic and robust framework for the enrichment of tabular datasets using unstructured data. Our approach utilizes transformer-based, pre-trained Language Models (LM) (Devlin et al., 2018) to identify and prioritize promising candidate features in the external data source. FeSTE then applies a novel process of ana- lyzing the relationships between the unstructured features and the dataset’s target class values, and automatically generating new tabular features. To overcome the difficulty imposed by datasets of …

Content truncated to 500 words