Joint Multi-Label Attention Networks for Social Text Annotation

Proceedings of NAACL-HLT 2019, pages 1348–1354 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1348 Joint Multi-Label Attention Networks for Social Text Annotation Hang Dong1,2, Wei Wang2, Kaizhu Huang3, and Frans Coenen1 1Dept. of Computer Science, University of Liverpool 2Dept. of Computer Science and Software Engineering, Xi’an Jiaotong-Liverpool University 3Dept. of Electrical and Electronic Engineering, Xi’an Jiaotong-Liverpool University {HangDong,Coenen}@liverpool.ac.uk {Wei.Wang03,Kaizhu.Huang}@xjtlu.edu.cn Abstract We propose a novel attention network for doc- ument annotation with user-generated tags. The network is designed according to the hu- man reading and annotation behaviour. Usu- ally, users try to digest the title and obtain a rough idea about the topic first, and then read the content of the document. Present research shows that the title metadata could largely affect the social annotation. To bet- ter utilise this information, we design a frame- work that separates the title from the con- tent of a document and apply a title-guided attention mechanism over each sentence in the content. We also propose two semantic- based loss regularisers that enforce the output of the network to conform to label semantic- s, i.e. similarity and subsumption. We anal- yse each part of the proposed system with t- wo real-world open datasets on publication and question annotation. The integrated ap- proach, Joint Multi-label Attention Network (JMAN), significantly outperformed the Bidi- rectional Gated Recurrent Unit (Bi-GRU) by around 13%-26% and the Hierarchical Atten- tion Network (HAN) by around 4%-12% on both datasets, with around 10%-30% reduction of training time. 1 Introduction Social annotation, or tagging, is a popular func- tionality allowing users to assign “keywords” to online resources for better semantic search and recommendation (Vander Wal, 2007; Singer et al., 2014; Gedikli and Jannach, 2014). Common so- cially annotated textual resources include ques- tions, papers, (micro-)blogs, product reviews, etc. In practice, however, only a limited number of re- sources is annotated with tags. Annotating a large number of documents requires much cognitive ef- fort and can be time-consuming. This has driven research on document annotation based on exist- ing tag sets (Bel´em et al., 2017; Nie et al., 2014). Recent studies formalise the automated so- cial annotation task as a multi-label classification problem (Gibaja and Ventura, 2015) and apply deep learning approaches (Li et al., 2016; Huang et al., 2016; Hassan et al., 2018). A strong base- line is the use of Bi-directional RNN (Schuster and Paliwal, 1997) with GRU (Cho et al., 2014) or LSTM (Hochreiter and Schmidhuber, 1997). Another more recent improvement is achieved through Hierarchical Attention Network (HAN) (Yang et al., 2016) which discriminates importan- t words and sentences from others, as adapted in (Hassan et al., 2018) for annotation. These mod- els, however, suffer from two issues: (i) simply s- canning over the words and sentences, the models do not fully mimic the way users read and anno- tate documents, and (ii) semantic relations, simi- larity and subsumption, among the labels are not considered. Our model focuses on simulating users’ read- ing and annotation behaviour …

Content truncated to 500 words