Trivia Score and Ranking Estimation Using Support Vector Regression and RankNet

Trivia Score and Ranking Estimation Using Support Vector Regression and RankNet Kazuya Niina Kazutaka Shimada Kyushu Institute of Technology 680-4 Kawazu Iizuka Fukuoka 820-8502 Japan {k niina, shimada}@pluto.ai.kyutech.ac.jp Abstract Dialogue systems have been increasingly im- portant these days. In particular, non-task- oriented dialogue systems are studied because of the success of neural network approaches such as seq2seq models. However, these mod- els tend to generate simple responses such as “yes” and “ok.” To construct a dialogue sys- tem that holds users’ attention continuously, we need to generate utterances that capture the interest of the user. In this paper, we propose a method to extract trivia sentences for the purpose. Trivia information perhaps adds a surprise to users. Therefore, captur- ing trivia information is beneficial for dia- logue systems. We estimate a trivia score of a sentence by using machine learning ap- proaches, Support Vector Regression (SVR) and RankNet. We obtained 0.79 and 0.78 on SVR for the nDCG@5 and RankNet for the nDCG@10, respectively. We focus on the subject word in each sentence. The method with subject information outperformed that without subject information; 0.79 with subject information vs. 0.64 without subject informa- tion on the SVR for the nDCG@5. 1 Introduction Dialogue systems, such as Siri and Alexa, have been increasingly important these days. In particular, non-task-oriented dialogue systems are studied be- cause of the success of neural networks or reinforce- ment learning approaches such as seq2seq models (Vinyals and Le, 2015; Li et al., 2016). However, these models tend to generate simple responses such as “yes” and “ok”. Therefore, users often get bored with the conversation based on such dialogue sys- tems. To solve this problem, we need to generate utterances that stimulate users’ interest. In this paper, we focus on trivia to solve the prob- lem. Here, we define trivia information as a lesser- known and interesting fact. For example, the fol- lowing sentence is trivia information; “If red swamp crayfishes eat mackerels, the body color becomes blue.” We believe that the trivia sentences can cap- ture the interest of the users and are beneficial to construct a dialogue system that holds users’ atten- tion continuously. To identify trivia information, we propose a method for estimating a trivia score of each sen- tence. In the method, we focus on a relation be- tween the main topic and words in each sentence. We extract the main topic from a target sentence and then identify an important noun for estimating the trivia score. We compute feature values from the word pair and then apply them to machine learning approaches. We use the Support Vector Regression (SVR) and the RankNet as the machine learning ap- proach. 2 Related Work Prakash et al. (2015) have proposed a mining method for interesting trivia. They used trivia infor- mation in IMDB as the training data. They extracted named entities and superlative words as features for a machine learning method. They estimated inter- estingness as trivia by using SVMrank. Fatma et al. (2017) have proposed …

Content truncated to 500 words