Text Similarity Estimation Based on Word Embeddings and Matrix Norms for Targeted Marketing

Proceedings of NAACL-HLT 2019, pages 1827–1836 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1827 Text Similarity Estimation Based on Word Embeddings and Matrix Norms for Targeted Marketing Tim vor der Br¨uck School of Information Technology Lucerne University of Applied Sciences and Arts Switzerland tim.vorderbrueck@hslu.ch Marc Pouly School of Information Technology Lucerne University of Applied Sciences and Arts Switzerland marc.pouly@hslu.ch Abstract The prevalent way to estimate the similarity of two documents based on word embeddings is to apply the cosine similarity measure to the two centroids obtained from the embed- ding vectors associated with the words in each document. Motivated by an industrial appli- cation from the domain of youth marketing, where this approach produced only mediocre results, we propose an alternative way of com- bining the word vectors using matrix norms. The evaluation shows superior results for most of the investigated matrix norms in compari- son to both the classical cosine measure and several other document similarity estimates. 1 Introduction Estimating semantic document similarity is of ut- most importance in a lot of different areas, like plagiarism detection, information retrieval, or text summarization. We will focus here on an NLP application that has been less researched, i.e., the assignment of people to the best matching target group to allow for running precise and customer- oriented marketing campaigns. Until recently, similarity estimates were pre- dominantly based either on deep semantic ap- proaches or on typical information retrieval tech- niques like Latent Semantic Analysis. In the last couple of years, however, so-called word and sen- tence embeddings became state-of-the-art. The prevalent approach to document similarity estimation based on word embeddings consists in measuring the similarity between the vector rep- resentations of the two documents derived as fol- lows: 1. The word embeddings (often weighted by the tf-idf coefficients of the associated words (Brokos et al., 2016)) are looked up in a hashtable for all the words in the two doc- uments to compare. These embeddings are determined beforehand on a very large cor- pus typically using either the skip gram or the continuous bag of words variant of the Word2Vec model (Mikolov et al., 2013). 2. The centroid over all word embeddings be- longing to the same document is calculated to obtain its vector representation. If vector representations of the two documents to compare were successfully established, a similar- ity estimate can be obtained by applying the cosine measure to the two vectors. Let x1, . . . , xm and y1, . . . , yn be the word vec- tors of two documents. The cosine similarity value between the two document centroids C1 und C2 is given by: cos(∠( 1 m m � i=1 xi, 1 n n � i=1 yi)) = �m i=1 �n j=1⟨xi, yj⟩ mn∥C1∥∥C2∥ (1) Hence, potentially small values of ⟨xi, yj⟩ can have in aggregate a considerable influence on the total similarity estimate, which makes this esti- mate vulnerable to noise in the data. We propose an alternative approach …

Content truncated to 500 words