Improving Robustness of Language Models from a Geometry-aware Perspective