Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information