Planning vs Reasoning: Ablations to Test Capabilities of LoRA layers
This work addresses the problem of understanding and improving efficient fine-tuning for AI models, particularly for researchers and practitioners in machine learning, though it is incremental as it builds on existing LoRA methods with new evaluations.
This paper investigated whether LoRA layers can enhance reasoning and planning abilities in models, finding through ablation studies on GPT-2 that reasoning capabilities are primarily in low-rank spaces and can be effectively boosted with LoRA, with a 2-3x lower rank requirement for reasoning compared to planning tasks.
Low-Rank Adaptation (LoRA) layers have emerged as a promising approach for efficient model fine-tuning, but their capabilities and limitations have not been fully explored. This paper: 1) Investigates the fundamental question of whether LoRA layers are effective at increasing reasoning + planning abilities 2) We introduce HashChain Reasoning, a novel evaluation dataset that deterministically tests reasoning capabilities. Through systematic ablation studies on GPT-2, we demonstrate that reasoning capabilities appear to exist primarily in low-rank spaces and can be effectively enhanced using LoRA layers. The effective rank analysis of trained LoRA matrices reveals a 2-3x lower rank requirement for reasoning tasks compared to planning tasks, giving context on where LoRA layers would be effective. This also provides evidence for reasoning fundamentally preferring low-parameter spaces for generalization.