Using Human Attention to Extract Keyphrase from Microblog Post

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5867–5872 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5867 Using Human Attention to Extract Keyphrase from Microblog Post Yingyi Zhang Chengzhi Zhang∗ Nanjing University of Science and Technology {yingyizhang, zhangcz}@njust.edu.cn Abstract This paper studies automatic keyphrase ex- traction on social media. Previous works have achieved promising results on it, but they neglect human reading behavior during keyphrase annotating. The human attention is a crucial element of human reading behavior. It reveals the relevance of words to the main topics of the target text. Thus, this paper aims to integrate human attention into keyphrase extraction models. First, human attention is represented by the reading duration estimated from eye-tracking corpus. Then, we merge human attention with neural network models by an attention mechanism. In addition, we also integrate human attention into unsuper- vised models. To the best of our knowledge, we are the first to utilize human attention on keyphrase extraction tasks. The experimental results show that our models have significant improvements on two Twitter datasets. 1 Introduction Rapidly growth of user-generated content on so- cial media has far outpaced human beings’ read- ing and understanding capacity. Keyphrase ex- traction is one of the technologies that can or- ganize this massive content. A keyphrase con- sists of one or more salient words, which repre- sents the main topics of a document. It has a se- ries of downstream applications, e.g., text summa- rization (Zhao et al., 2011a) and information re- trieval (Choi et al., 2012). Generally, corpus with human annotated keyphrases are needed to train models in su- pervised keyphrase extraction frameworks. The premise for annotators to annotate keyphrases is to read the corresponding content. Intuitively, features estimated from human reading behavior can be leveraged to assist keyphrase extraction. *Corresponding Author. Previous studies on keyphrase extraction have ignored these features (Zhang et al., 2016, 2018). Thus, this paper aims to integrate the reading behavior into keyphrase extraction frameworks. When human reading, they do not pay the same attention to all words (Carpenter and Just, 1983). The reading time of per-word is the indicative of textual (as well as lexical, syntactic and seman- tic) processing (Demberg and Keller, 2008), which reflects human attention on various content. To obtain human attention during reading, this paper estimates eye fixation duration from eye-tracking corpus inspired by Carpenter and Just (1983) and Barrett et al. (2018). The modern-day eye track- ing equipment resulting in a very rich and detailed dataset (Cop et al., 2017). Thus, we utilize open- source eye-tracking corpora and do not require eye-tracking information of the target datasets. To integrate human attention into keyphrase ex- traction models, this paper constructs a neural net- work model with attention mechanism. Attention mechanism is a neural module designed to imi- tate human visual attention when they reading and looking (Bahdanau et al., 2014). To regularize the predicted value of attention mechanism, hu- man attention estimated from eye-tracking corpus …

Content truncated to 500 words