KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction