Enhancing Multilingual Latent Semantic Analysis with Term Alignment Information

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 49–56 Manchester, August 2008 Enhancing Multilingual Latent Semantic Analysis with Term Alignment Information Brett W. Bader Computer Science & Informatics Department Sandia National Laboratories P. O. Box 5800, MS 1318 Albuquerque, NM 87185-1318, USA bwbader@sandia.gov Peter A. Chew Cognitive Systems Research & Applications Department Sandia National Laboratories P. O. Box 5800, MS 1011 Albuquerque, NM 87185-1011, USA pchew@sandia.gov Abstract Latent Semantic Analysis (LSA) is based on the Singular Value Decompo- sition (SVD) of a term-by-document matrix for identifying relationships among terms and documents from co- occurrence patterns. Among the multi- ple ways of computing the SVD of a rectangular matrix X, one approach is to compute the eigenvalue decomposition (EVD) of a square 2 × 2 composite ma- trix consisting of four blocks with X and XT in the off-diagonal blocks and zero matrices in the diagonal blocks. We point out that significant value can be added to LSA by filling in some of the values in the diagonal blocks (corre- sponding to explicit term-to-term or document-to-document associations) and computing a term-by-concept ma- trix from the EVD. For the case of mul- tilingual LSA, we incorporate information on cross-language term alignments of the same sort used in Sta- tistical Machine Translation (SMT). Since all elements of the proposed EVD-based approach can rely entirely on lexical statistics, hardly any price is paid for the improved empirical results. In particular, the approach, like LSA or SMT, can still be generalized to virtu- ally any language(s); computation of the EVD takes similar resources to that of the SVD since all the blocks are sparse; © 2008. Licensed under the Creative Commons Attribution- Noncommercial-Share Alike 3.0 Unported license (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. and the results of EVD are just as eco- nomical as those of SVD. 1 Introduction It is close to two decades now since Deerwester et al. (1990) first proposed the application of the Singular Value Decomposition (SVD) to term- by-document arrays as a statistics-based way of representing how terms and documents fit to- gether within a semantic space. Since the ap- proach was supposed to ‘get beyond’ the terms themselves to their underlying semantics, the approach became known as Latent Semantic Analysis (LSA). Soon after this application of SVD was widely publicized, it was suggested by Berry et al. (1994) that, with a parallel corpus, the approach could be extended to pairs of languages to allow cross-language information retrieval (IR). It has since been confirmed that LSA can be applied not just to pairs of languages, but also simultane- ously to groups of languages, again given the existence of a multi-parallel corpus (Chew and Abdelali 2007). In this paper, we return to the basics of LSA by examining its relationship with SVD, and, in turn, the mathematical relationship of SVD to the eigenvalue decomposition (EVD). These details are discussed in section 2. It has previously been suggested (for example, in Hendrickson 2007) that IR results could be improved by …

Content truncated to 500 words