Towards Unsupervised Text Classification Leveraging Experts and Word Embeddings

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 371–379 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 371 Towards Unsupervised Text Classification Leveraging Experts and Word Embeddings Zied Haj-Yahia Capgemini Invent zied.haj-yahia@capgemini.com Adrien Sieg Capgemini Invent adrien.sieg@capgemini.com L´ea A. Deleris BNP Paribas lea.deleris@bnpparibas.com Abstract Text classification aims at mapping documents into a set of predefined categories. Super- vised machine learning models have shown great success in this area but they require a large number of labeled documents to reach adequate accuracy. This is particularly true when the number of target categories is in the tens or the hundreds. In this work, we explore an unsupervised approach to classify documents into categories simply described by a label. The proposed method is inspired by the way a human proceeds in this situation: It draws on textual similarity between the most relevant words in each document and a dic- tionary of keywords for each category reflect- ing its semantics and lexical field. The nov- elty of our method hinges on the enrichment of the category labels through a combination of human expertise and language models, both generic and domain specific. Our experiments on 5 standard corpora show that the proposed method increases F1-score over relying solely on human expertise and can also be on par with simple supervised approaches. It thus provides a practical alternative to situations where low- cost text categorization is needed, as we illus- trate with our application to operational risk incidents classification. 1 Introduction Document classification is a standard task in ma- chine learning (Joachims, 1999; Sebastiani, 2002). Its applications span a variety of ”use cases and contexts, e.g., email filtering, news article clus- tering, clinical document classification, expert- question matching”. The standard process for text categorization relies on supervised and semi- supervised approaches. The motivation for the present effort comes from the banking sector, in particular the manage- ment of operational risks. This category of risks corresponds to the broad set of incidents that are neither credit nor market risk and includes issues related to internal and external fraud, cybersecu- rity, damages on physical assets, natural disasters, etc. The practical management of operational risk is partially based on the management of a dataset of historical operational risk incidents where each incident is described in details and that is shared on a regular basis with regulators. Historically, all incident reports have been mapped to about twenty categories of risk issued from the regulator. However, from an operational perspective, a higher number of risk categories is relevant to better capture the nuances around the incidents and enable relevant comparisons. This led to the creation of a new internal risk taxon- omy of risk composed of 264 categories, each de- scribed by a label (a few words). To make it op- erational, the stock of all internal and external in- cident reports had to be classified into categories from the new internal taxonomy. However, since it had never been used before, …

Content truncated to 500 words