LiFT: Does Instruction Fine-Tuning Improve In-Context Learning for Longitudinal Modelling by Large Language Models?
For researchers working on longitudinal NLP tasks (e.g., tracking behavior/opinion changes over time), LiFT provides a method to enhance LLM performance on rare change events and out-of-distribution data, though it is incremental over existing instruction fine-tuning approaches.
LiFT, a longitudinal instruction fine-tuning framework, improves in-context learning for longitudinal NLP tasks by unifying diverse tasks under a shared instruction schema with a curriculum that increases temporal difficulty. Across models from 1B to 14B parameters, LiFT consistently outperforms base-model ICL, with strong gains on out-of-distribution data and minority change events.
Longitudinal NLP tasks require reasoning over temporally ordered text to detect persistence and change in human behavior and opinions. However, in-context learning with large language models struggles on tasks where models must integrate historical context, track evolving interactions, and handle rare change events. We introduce LiFT, a longitudinal instruction fine-tuning framework that unifies diverse longitudinal modeling tasks under a shared instruction schema. LiFT uses a curriculum that progressively increases temporal difficulty while incorporating few-shot structure and temporal conditioning to encourage effective use of past context. We evaluate LiFT across five datasets. Models trained on longitudinal tasks with different levels of temporal granularity are tested for generalisability on two separate datasets. Across models with different parameter sizes (OLMo (1B/7B), LLaMA-8B, and Qwen-14B), LiFT consistently outperforms base-model ICL, with strong gains on out-of-distribution data and minority change events.