Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 436–447 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 436 KOMPETENCER: Fine-grained Skill Classification in Danish Job Postings via Distant Supervision and Transfer Learning Mike Zhang*♢ Kristian Nørgaard Jensen*♢ Barbara Plank♢♣ ♢Department of Computer Science, IT University of Copenhagen, Denmark ♣Center for Information and Language Processing (CIS), LMU Munich, Germany {mikz, krnj}@itu.dk bplank@cis.uni-muenchen.de Abstract Skill Classification (SC) is the task of classifying job competences from job postings. This work is the first in SC applied to Danish job vacancy data. We release the first Danish job posting dataset: KOMPETENCER (en: competences), annotated for nested spans of competences. To improve upon coarse-grained annotations, we make use of The European Skills, Competences, Qualifications and Occupations (ESCO; le Vrang et al. (2014)) taxonomy API to obtain fine-grained labels via distant supervision. We study two setups: The zero-shot and few-shot classification setting. We fine-tune English-based models and RemBERT (Chung et al., 2020) and compare them to in-language Danish models. Our results show RemBERT significantly outperforms all other models in both the zero-shot and the few-shot setting. Keywords: Skill Classification, Distant Supervision, Transfer Learning, Domain Adaptive Pretraining, Job Postings 1. Introduction Job Posting data (JPs) is emerging on a variety of plat- forms in big quantities, and can provide insights on la- bor market skill set demands and aid job matching (Ba- log et al., 2012). Skill Classification (SC) is to classify competences (i.e., hard and soft skills) necessary for any occupation from unstructured text or JPs. Several works focus on Skill Identification (Jia et al., 2018; Sayfullina et al., 2018; Tamburri et al., 2020). This is to classify whether a skill occurs in a sentence or job description. However, continuing the pipeline, there is little work in further categorizing the identified skills by leveraging taxonomies such as ESCO. An- other limitation is the scope of language, where all pre- vious work focus on English job postings. This hinders in particular local job seekers from finding an occupa- tion suitable to their specific skills within their commu- nity via online job platforms. In this work, we look into the Danish labor market. We introduce KOMPETENCER, a novel Danish job post- ing dataset annotated on the span-level for nested Skill and Knowledge Components (SKCs) in job postings. We do not directly annotate for the fine-grained taxon- omy codes from e.g., ESCO, but rather annotate more generic spans of SKCs (Figure 2), and then exploit the ESCO API to bootstrap fine-grained SKCs via distant supervision (Mintz et al., 2009) and create “silver” data for skill classification. Our proposed distant supervi- sion pipeline is denoted in Figure 1. Recently, Natural Language Processing has seen a surge of several transfer learning methods and architec- ture which help improve state-of-the-art significantly on several tasks (Peters et al., 2018; Howard and Ruder, 2018; Radford et al., 2018; Devlin et al., 2019). In this work, we explore the benefits of zero-shot cross- lingual transfer …

Content truncated to 500 words