Featureless Domain-Specific Term Extraction with Minimal Labelled Data

Featureless Domain-Specific Term Extraction with Minimal Labelled Data Rui Wang The University of Western Australia 35 Stirling Highway, Crawley, W.A. 6009, Australia rui.wang@research. uwa.edu.au Wei Liu∗ The University of Western Australia 35 Stirling Highway, Crawley, W.A. 6009, Australia wei.liu@uwa.edu.au Chris McDonald The University of Western Australia 35 Stirling Highway, Crawley, W.A. 6009, Australia chris.mcdonald@ uwa.edu.au Abstract Supervised domain-specific term extrac- tion often suffers from two common prob- lems, namely labourious manual feature selection, and the lack of labelled data. In this paper, we introduce a weakly super- vised bootstrapping approach using two deep learning classifiers. Each classifier learns the representations of terms sep- arately by taking word embedding vec- tors as inputs, thus no manually selected feature is required. The two classifiers are firstly trained on a small set of la- belled data, then independently make pre- dictions on a subset of the unlabeled data. The most confident predictions are subse- quently added to the training set to retrain the classifiers. This co-training process minimises the reliance on labelled data. Evaluations on two datasets demonstrate that the proposed co-training approach achieves a competitive performance with limited training data as compared to stan- dard supervised learning baseline. 1 Introduction Domain-specific terms are essential for many knowledge management applications, such as clinical text processing, risk management, and equipment maintenance. Domain-specific term extraction aims to automatically identify domain relevant technical terms that can be either uni- gram words or multi-word phrases. Supervised domain-specific term extraction often relies on the training of a binary classifier to identify whether or not a candidate term is relevant to the do- main (da Silva Conrado et al., 2013; Foo and Merkel, 2010; Nazar and Cabr´e, 2012). In such approaches, term extraction models are built upon ∗Corresponding author manually selected features including the local sta- tistical and linguistic information (e.g. frequency, co-occurrence frequency, or linguistic patterns), and external information form third-party knowl- edge bases (e.g. WordNet, DBpedia). Designing and evaluating different feature combinations turn the development of term extraction models into a time-consuming and labor-intensive exercise. In addition, these approaches require a large amount of labelled training data to generalise the learning. However, labelled data is often hard or impractical to obtain. In this paper, our first objective is to minimise the usage of labelled data by training two clas- sifiers in a co-training fashion. Co-training is a weakly supervised learning mechanism intro- duced by Blum and Mitchell (1998), which tack- les the problem of building a classification model from a dataset with limited labelled data among the majority of unlabelled ones. It requires two classifiers, each built upon separate views of the data. Each view represents a separate set of man- ually selected features that must be sufficient to learn a classifier. For example, Blum and Mitchell classify web pages based on words appearing in the content of a web page, and words in hyper- links pointing to the web page. Co-training starts with training each classifier on a small labelled dataset, then each classifier is used …

Content truncated to 500 words