Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization