Integrating Semantic Knowledge to Tackle Zero-shot Text Classification

Proceedings of NAACL-HLT 2019, pages 1031–1040 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1031 Integrating Semantic Knowledge to Tackle Zero-shot Text Classification Jingqing Zhang ∗ Data Science Institute Imperial College London London, UK Piyawat Lertvittayakumjorn ∗ Department of Computing Imperial College London London, UK {jingqing.zhang15,pl1515,y.guo}@imperial.ac.uk Yike Guo Data Science Institute Imperial College London London, UK Abstract Insufficient or even unavailable training data of emerging classes is a big challenge of many classification tasks, including text classifica- tion. Recognising text documents of classes that have never been seen in the learning stage, so-called zero-shot text classification, is there- fore difficult and only limited previous works tackled this problem. In this paper, we pro- pose a two-phase framework together with data augmentation and feature augmentation to solve this problem. Four kinds of semantic knowledge (word embeddings, class descrip- tions, class hierarchy, and a general knowl- edge graph) are incorporated into the pro- posed framework to deal with instances of un- seen classes effectively. Experimental results show that each and the combination of the two phases achieve the best overall accuracy com- pared with baselines and recent approaches in classifying real-world texts under the zero- shot scenario. 1 Introduction As one of the most fundamental problems in ma- chine learning, automatic classification has been widely studied in several domains. However, many approaches, proven to be effective in tradi- tional classification tasks, cannot catch up with a dynamic and open environment where new classes can emerge after the learning stage (Romera- Paredes and Torr, 2015). For example, the number of topics on social media is growing rapidly, and the classification models are required to recognise the text of the new topics using only general in- formation (e.g., descriptions of the topics) since labelled training instances are unfeasible to ob- tain for each new topic (Lee et al., 2011). This scenario holds in many real-world domains such ∗ Piyawat Lertvittayakumjorn and Jingqing Zhang con- tributed equally to this project. as object recognition and medical diagnosis (Xian et al., 2017; World Health Organization, 1996). Zero-shot learning (ZSL) for text classification aims to classify documents of classes which are absent from the learning stage. Although it is challenging for a machine to achieve, humans are able to learn new concepts by transferring knowl- edge from known to unknown domains based on high-level descriptions and semantic representa- tions (Thrun and Pratt, 1998). Therefore, without labelled data of unseen classes, a zero-shot learn- ing framework is expected to exploit supportive semantic knowledge (e.g., class descriptions, rela- tions among classes, and external domain knowl- edge) to generally infer the features of unseen classes using patterns learned from seen classes. So far, three main types of semantic knowl- edge have been employed in general zero-shot sce- narios (Fu et al., 2018). The most widely used one is semantic attributes of classes such as vi- sual concepts (e.g., colours, shapes) and semantic properties (e.g., behaviours, functions) (Lampert et al., 2009; Zhao et al., 2018). The second …

Content truncated to 500 words