Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation